Generating georeference information for aerial images

ABSTRACT

Systems and methods are disclosed for generating georeference information with respect to aerial images. In particular, in one or more embodiments, systems and methods generate georeference information relating to aerial images captured without ground control points based on existing aerial images. For example, systems and methods can access a new set of aerial images without ground control points and utilize existing aerial images containing ground control points to generate a georeferenced representation corresponding to the features of the new set of aerial images. Similarly, systems and methods can access a new image without ground control points and utilize an existing georeferenced orthomap to produce a georeferenced orthomap corresponding to the features of the new image. One or more embodiments of the disclosed systems and methods permit users to obtain georeference information related to new images without the need to place ground control points or collect additional georeference information.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. application Ser. No.14/857,238, filed Sep. 17, 2015. The aforementioned application ishereby incorporated by reference in its entirety.

BACKGROUND 1. Technical Field

One or more embodiments of the present disclosure relate generally togeoreference information. More specifically, one or more embodiments ofthe present disclosure relate to systems and methods for generatinggeoreference information for aerial images.

2. Background and Relevant Art

Recent years have seen a rapid increase in the use of digital aerialimages in various applications. In particular, given the recentproliferation of unmanned aerial vehicles (UAVs) in commercial andindustrial applications, digital aerial photography has become morecommon in individual and business environments. For instance,individuals now commonly utilize UAVs to capture digital aerial imagesof homes, places of interest, or even recreational activities.

In many applications, however, individuals and businesses find itimperative to tie digital aerial image data to precise geographiclocations. For example, in a construction application, a design companymay find it critical to determine the location of structures as theyexist in situ and/or the location of structures progressing duringconstruction. Indeed, many applications require digital aerial imagedata that corresponds to accurate real-world coordinates with onlycentimeters of permissible variance.

To accomplish this objective, many common digital image systems captureimages tied to known georeference locations. For instance, common imagesystems place visible ground control points (i.e., markers) at knowngeographic locations before capturing digital image data. Common imagesystems then determine the precise geographic location of featureswithin captured images based on these known points.

Although systems that utilize markers permit users to relate digitalimage data to known ground control points with remarkable accuracy, suchsystems introduce their own problems. For instance, businesses commonlyneed updated aerial imagery of a site over a period of days, weeks,months, or years. Common systems require users to place, or maintain,ground control points each time users seek to capture digital aerialimagery. Thus, for example, a business operating an active constructionsite may desire updated digital aerial image data related toconstruction progress at the end of each week, requiring users to placemarkers every week to produce accurate georeferenced image data. Becauseplacing the markers may require specialized expertise and surveyingequipment, this process can result in excessive costs in both time andexpense.

Moreover, in some instances, users may forget to place markers (or mayinaccurately place markers) when capturing aerial image data. Suchsituations may result in additional wasted time and expense, as usersare required to recapture image data or proceed without accurategeoreference information. Furthermore, in some applications, it may notbe possible to access a site to replace or maintain ground controlpoints. For example, in taking digital aerial images at remote locations(or in locations that experience extreme temperatures or other weatherconditions) it may be impossible, or impractical, to repeatedlyestablish ground control points or maintain ground control points overtime.

Accordingly, a number of problems and disadvantages exist withconventional systems for creating georeferenced digital aerial imagesusing temporary ground control points.

BRIEF SUMMARY

Embodiments of the present disclosure provide benefits and/or solve oneor more of the foregoing or other problems in the art with systems andmethods that generate georeference information with regard to aerialimages. For instance, one or more embodiments include systems andmethods that utilize existing aerial images with georeferenceinformation (e.g., ground control points) to create georeferenceinformation for aerial images captured without such information. Inparticular, in one or more embodiments, systems and methods use variousmeans to compare features of aerial images captured without georeferenceinformation to images captured with georeference information, and creategeoreference information with regard to the new aerial images.

For example, one or more embodiments include systems and methods thataccess a first set of aerial images of a site showing known groundcontrol points. From the first set of images, systems and methods canbuild an initial unreferenced three-dimensional representation andgenerate a transformation to the known ground control points. Systemsand methods can then access new aerial images of the site that do notshow known ground control points. In one or more embodiments, thedisclosed systems and methods utilize the new aerial images and theinitial unreferenced three-dimensional representation to build a newunreferenced three-dimensional representation. The disclosed systems andmethods can then apply the generated transformation to the newunreferenced three-dimensional representation to create a newgeoreferenced three-dimensional representation. In one or moreembodiments, systems and methods utilize the new georeferencedthree-dimensional representation to generate a georeferencedthree-dimensional model or use the new georeferenced three-dimensionalrepresentation to create a georeferenced orthomap of the site.

The disclosed systems and methods permit users to avoid the time andexpense associated with repeatedly placing or maintaining ground controlpoints at a site over time. Because the disclosed systems and methodscan generate georeference information for aerial images based onexisting image data, users can capture aerial images containinggeoreference information (e.g., ground control points) a single time,capture new aerial images without georeference information at subsequenttimes, and still correlate features within the new aerial images toprecise geographic locations.

The disclosed systems and methods permit users to obtain georeferenceinformation for images captured (either purposefully or unintentionally)without ground control points (or GPS information). For example, if inutilizing a UAV to capture aerial images, a user accidentally fails toplace ground control points (or inaccurately records the location ofground control points), the disclosed systems and methods permit theuser to correlate the captured image data with a precise location onEarth.

Moreover, the disclosed systems and methods provide accurategeoreference information. For instance, utilizing one or more of thedisclosed systems and methods, users can correlate aerial image data toreal-world geographic coordinates within centimeters of accuracy,despite the fact that aerial images were not originally captured withgeoreference information. Systems and methods can accomplish theseresults despite changes to ground conditions, changes in illumination,changes in perspective amongst the various images, variations in cameraparameters, changes in camera location (e.g., elevation, aeriallocation, etc.), and other changes.

Additional features and advantages of exemplary embodiments of thepresent disclosure will be set forth in the description which follows,and in part will be obvious from the description, or may be learned bythe practice of such exemplary embodiments. The features and advantagesof such embodiments may be realized and obtained by means of theinstruments and combinations particularly pointed out in the appendedclaims. These and other features will become more fully apparent fromthe following description and appended claims, or may be learned by thepractice of such exemplary embodiments as set forth hereinafter. Theforegoing summary is not an extensive overview, and it is not intendedto identify key elements or indicate a scope. Rather the foregoingsummary identifies aspects of embodiments as a prelude to the detaileddescription presented below.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above recited and otheradvantages and features of the invention can be obtained, a moreparticular description of the invention briefly described above will berendered by reference to specific embodiments thereof that areillustrated in the appended drawings. It should be noted that thefigures are not drawn to scale, and that elements of similar structureor function are generally represented by like reference numerals forillustrative purposes throughout the figures. Understanding that thesedrawings depict only typical embodiments of the invention and are nottherefore to be considered to be limiting of its scope, the inventionwill be described and explained with additional specificity and detailthrough the use of the accompanying drawings in which:

FIG. 1 illustrates a schematic diagram of an image georeferencing systemin accordance with one or more embodiments;

FIG. 2 illustrates a schematic diagram of an exemplary environment inwhich the image georeferencing system of FIG. 1 can operate inaccordance with one or more embodiments;

FIG. 3A illustrates a representation of one or more aerial imagesaccessed in accordance with one or more embodiments;

FIG. 3B illustrates a representation of an unreferencedthree-dimensional representation generated in accordance with one ormore embodiments;

FIG. 3C illustrates a representation of a transformation applied to theunreferenced three-dimensional representation of FIG. 3B.

FIG. 3D illustrates a representation of one or more new aerial imagesaccessed in accordance with one or more embodiments;

FIG. 3E illustrates a representation of a new unreferencedthree-dimensional representation generated in accordance with one ormore embodiments;

FIG. 3F illustrates a representation of the transformation of FIG. 3Capplied to the new unreferenced three-dimensional representation of FIG.3E in accordance with one or more embodiments;

FIG. 4A illustrates a representation of observed positions of groundcontrol points in a three-dimensional representation in accordance withone or more embodiments;

FIG. 4B illustrates a representation of a transformation betweenobserved positions of ground control points in a three-dimensionalrepresentation to measured positions of ground control points inaccordance with one or more embodiments;

FIG. 4C illustrates applying a transformation inversely to measuredpositions of ground control points in accordance with one or moreembodiments;

FIG. 5 illustrates a representation of generating georeferenceinformation for one or more new images in accordance with one or moreembodiments;

FIG. 6 illustrates another representation of generating georeferenceinformation for one or more new images in accordance with one or moreembodiments;

FIG. 7 illustrates a representation of a UAV navigating based ongeoreference information generated in accordance with one or moreembodiments;

FIG. 8A illustrates a representation of a georeferenced orthomap anddesign elements in accordance with one or more embodiments;

FIG. 8B illustrates a representation of a modified georeferencedorthomap and design elements in accordance with one or more embodiments;

FIG. 9 illustrates a flowchart of a series of acts in a method ofcreating georeference information in accordance with one or moreembodiments;

FIG. 10 illustrates another flowchart of a series of acts in a method ofcreating georeference information in accordance with one or moreembodiments;

FIG. 11 illustrates a block diagram of an exemplary computing device inaccordance with one or more embodiments.

DETAILED DESCRIPTION

One or more embodiments of the present disclosure include an imagegeoreferencing system that generates georeference information related toone or more aerial images. In particular, in one or more embodiments thegeoreferencing system utilizes existing aerial images captured withgeoreference information (e.g., ground control points) to generategeoreference information for aerial images captured without suchinformation. More specifically, the image georeferencing system utilizesvarious methods to compare features of existing aerial images withgeoreference information to the features of new aerial images capturedwithout georeference information to generate georeference informationrelated to the new aerial images.

For example, in one or more embodiments the image georeferencing systemaccesses a set of aerial images of a site showing known ground controlpoints and builds an unreferenced three-dimensional representation basedon the set of aerial images. The image georeferencing system thengenerates a transformation relating the unreferenced three-dimensionalrepresentation to the known ground control points. The imagegeoreferencing system then accesses a new set of aerial images of thesite without known ground control points. Utilizing the new set ofaerial images and the unreferenced three-dimensional representation, theimage georeferencing system builds a new unreferenced three-dimensionalrepresentation. The image georeferencing system then applies thegenerated transformation to the new unreferenced three-dimensionalrepresentation to create a new georeferenced three-dimensionalrepresentation. The image georeferencing system can utilize the newgeoreferenced three-dimensional representation, for example, to producea three-dimensional model or generate a georeferenced orthomap.

In addition, in one or more embodiments the image georeferencing systemcan utilize a georeferenced orthomap (e.g., from the embodiment justdescribed) to generate georeference information for one or moreadditional images captured without georeference information. Inparticular, the image georeferencing system can access a georeferencedorthomap and a second aerial image, match features from thegeoreferenced orthomap with features from the second aerial image, andcompute a homography relating the georeferenced orthomap and the secondaerial image. Utilizing the homography, the image georeferencing systemcan update the georeferenced orthomap based on the new aerial image.

Moreover, in one or more embodiments the image georeferencing system cangenerate ground control points for an image captured without groundcontrol points. For example, one or more embodiments of the imagegeoreferencing system access a set of aerial images of a site showingknown ground control points and a new aerial image of the site notshowing ground control points. The image georeferencing system comparesone or more features from the set of aerial images with one or morefeatures from the new aerial image. Based on the comparison, the imagegeoreferencing system generates georeference information (e.g., groundcontrol points) for the new aerial image.

By generating georeference information based on existing aerial images,the georeferencing system permits a user to accurately tie new aerialimages to a precise location on Earth, even when the new aerial imagesare captured without georeference information. Thus, users can obtainaccurate georeferenced images of a site over time, without repeatedlyplacing ground control points. Similarly, users can obtain accurategeoreferenced images where ground control points were accidentallyomitted, or inaccurately measured.

Advantageously, the image georeferencing system can account forvariations in perspective effect, optical characteristics, illumination,equipment, and ground conditions. For instance, in capturing aerialimages over time, the perspective (e.g., location, rotation, angle,viewpoint), illumination (e.g., brightness, color, lighting, etc.),camera calibration, and other optical characteristics will vary fromimage to image. Similarly, with regard to aerial images captured days,weeks, or months apart, the ground conditions can change substantiallywith regard to mobile items (e.g., cars, equipment, people, etc.),scenery (change of season, adding or removing landscaping, etc.),structural items (e.g., removal or construction of buildings, roads,etc.), or other conditions.

In one or more embodiments, the image georeferencing system accounts forchanges in ground conditions by utilizing one or more filters. Inparticular, the image georeferencing system can identify common featuresbetween images by applying a filter that identifies outliers and/orremoves features. Thus, for example, the image georeferencing system canfilter out cars, people, or other items that move between images.

Similarly, in one or more embodiments, the image georeferencing systemaccounts for variations in perspective effect, optical characteristics,illumination, and equipment by utilizing one or more algorithms. Forexample, the image georeferencing system can account for variations byutilizing a structure from motion algorithm and a bundle adjustmentalgorithm to generate one or more three-dimensional representations(e.g., point clouds). In other embodiments, the image georeferencingsystem accounts for variations by utilizing a homography to relate twoperspectives of a planar surface (e.g., representations of the samelocation in two images) while blending other visual variations.

Moreover, one or more embodiments of the image georeferencing systemreduce error accumulation (i.e., drift) with regard to three-dimensionalrepresentations. In particular, during sequential structure from motionprocesses, it is possible for errors to accumulate in three-dimensionalrepresentations (e.g., points in the three-dimensional representationcan drift from accurate positions relative to either local or real-worldcoordinates). One or more embodiments of the image georeferencing systemreduce such error by utilizing a constrained bundle adjustment. Inparticular, in one or more embodiments, the georeferencing systemrefines the accuracy of ground control points and applies an additionalbundle adjustment algorithm constrained by the refined ground controlpoints to correct for drift in three-dimensional representations.

To further assist a user in obtaining accurate geographic informationquickly, in one or more embodiments, the image georeferencing systemoperates during flight of a UAV. For example, a user can load anexisting georeferenced orthomap to the computer system of a UAV. Duringflight, the UAV can capture a new image and generate georeferenceinformation for the new image. Furthermore, in one or more embodiments,the UAV can utilize the georeference information with regard to the newimage. For example, the UAV can utilize the georeference information todetermine the location of the UAV, identify areas to capture additionalimages, and/or navigate the UAV during flight.

Individuals and businesses can utilize georeference informationgenerated by the image georeferencing system in a variety ofapplications. As just discussed, the image georeferencing system canutilize georeferenced images for navigation. In addition, the imagegeoreferencing system can utilize georeferenced images in constructionapplications. For example, the image georeferencing system can providegeoreferenced images for display to assist in generating site maps,creating construction schedules, identifying construction progress,estimating expected costs, locating site features, modifying a site ordesign, building or demolishing a structure, designing improvements,creating aerial image overlays for design plans, generating as-builtplans, comparing actual construction realities with designed plans, orany other construction related process that relies upon updated andaccurate mapping. Indeed, in design-build applications, engineering andconstruction professionals design improvements during the constructionprocess; thus, continuously updated georeferenced images can aid in theongoing engineering and construction process. Thus, the georeferenceimaging system can assist in decisions and policies regarding thetiming, pricing, scheduling, designing, progress, and completion ofconstruction projects.

Aside from construction, the image georeferencing system can alsoprovide georeferenced images or georeference information to assist indecisions and policies in the fields of mining, agriculture, wildfiresurveillance/mapping, land preservation, real estate, hydrology, landuse planning, surveillance, security, livestock monitoring, disasterrelief, archeology, and other fields. For instance, in open-facedmining, the image georeferencing system can be utilized for locatingtarget mining areas, tracking mining progress, estimating availablemining materials, and other activities.

As used herein, the term “georeference information” refers toinformation that relates image data to a known location relative to theEarth. In particular, georeference information can relate all or aportion of an image to coordinates relative to the Earth based on acoordinate system. For example, georeference information can includeinformation relating an image feature (or other part of an image) to alatitude, longitude, township, range, section, quarter section,quarter-quarter section, meridian, baseline, arpent, site reference,local reference, or other coordinate system or reference. Similarly,georeference information can comprise one or more directions and/ordistances from recognized survey landmarks, markers, structures, ormonuments, such as a section corner. In addition, georeferenceinformation can comprise one or more directions and/or distances from aknown location associated with a beacon, transmitter, or other devicecapable of transmitting information to a sensor (e.g., a sensorassociated with a UAV). Thus, georeference information can comprise adata point (e.g. in a point cloud) defined by coordinates relative to alocation on earth; ground control points shown on an orthomap definedrelative to a location on earth; or ground control points shown on ananother aerial image defined relative to a location on earth.Georeference information can include location information obtained fromany source, including a global positioning system (GPS), a groundcontrol point, a marker, a tag, a survey, or any combination thereof.For example, georeference information can include a point shown on animage together with survey data identifying the known location on Earthof the point. Similarly, georeference information can include GPSinformation related to the position of a UAV relative to the Earth atthe time a camera on board the UAV captures an image.

As just discussed, georeference information can include one or moreground control points. As used herein, the term “ground control points”refers to points on the surface of the Earth of known location. Forexample, ground control points can include temporary or permanentmarkers placed at known locations of the Earth. Ground control pointsmay be visible to a UAV during flight. A ground control point cancomprise any variety of markers, tags, or identifiers, including signs,painted marks, taped marks, mobile platforms, boards, or any othermarkers visible to a UAV during flight. In particular, a ground controlpoint can include a temporary marker that looks like an “X” placed onthe Earth such that it is visible from the air. The location of theground control point may be measured prior to capturing an image, aftercapturing an image, prior to placing a marker, or after placing amarker. For example, a surveyor may determine the location of aparticular spot utilizing surveying equipment, the surveyor may mark thespot with a temporary marker visible from the air, and a UAV may captureimages showing the temporary marker during flight.

As used herein, the term “site” refers to any portion of a geographicarea for which aerial image data is desired. For instance, a site mayinclude a geographic area containing geography, structures, vegetation,or other items of interest. For example, a site may refer to aconstruction site, a mining area, a field, a plot of real estate, adisaster area, or some other geographic region.

As used herein, the term “orthomap” or “orthophoto” refers to ageometrically corrected aerial image portraying a uniform scale. Inparticular, an orthomap can comprise a geometrically corrected aerialimage where the effects of tilt and relief have been removed. Because anorthomap portrays a uniform scale, it is possible to measure distancesdirectly from an orthomap. For example, a construction manager canutilize an orthomap of a construction site to accurately measure thedistance between objects appearing on the orthomap. Similarly, adesigner can overlay, and match, an orthomap to existing scaled designplans.

As used herein, the term “three-dimensional representation” refers toany digital data depicting a three-dimensional object or site. The termthree-dimensional representation includes a three-dimensional pointcloud, a three-dimensional mesh, a three-dimensional surface, or anyother representation derived from the observation of a point or landmarkfrom a single or multiple views.

As used herein, the term “point cloud” refers to a set of data points ina coordinate system. In particular, the term point cloud may refer to aset of data points on a three-dimensional coordinate system. Forinstance, a point cloud may refer to the set of data points on athree-dimensional coordinate system that represent the external surfaceof one or more objects. More specifically, a point cloud may refer tothe set of data points on a three-dimensional coordinate system thatrepresent a particular location of the Earth, a construction site, abuilding, a mine, and/or some other three-dimensional object or feature.

As used herein, the term “georeferenced” refers to data that isreferenced to a known location on the Earth. For instance, ageoreferenced orthomap refers to an orthomap that is referenced to aknown location on Earth (e.g., shows or is tied to a known groundcontrol point). Similarly, a georeferenced three-dimensionalrepresentation is a three-dimensional representation where the locationof the three-dimensional representation is tied to a known location onEarth (e.g., where points in a three-dimensional point cloud are definedin reference to real-world coordinates).

In contrast, as used herein, the term “unreferenced” refers to data thatis not referenced to a known location on the Earth. For example, anunreferenced orthomap comprises an orthomap that is not tied to groundcontrol points. Similarly, an unreferenced three-dimensionalrepresentation comprises a three-dimensional representation withthree-dimensional digital data not defined relative to a location onEarth (e.g., a point cloud whose coordinate system and/or data pointsare not defined relative to a location on Earth). To illustrate, a userof the image georeferencing system can utilize a set of images to createan unreferenced point cloud (i.e., a point cloud whose coordinates areconsistent relative to other images, thus creating a three-dimensionalrepresentation, but whose coordinates are not defined in relation to alocation on Earth). By utilizing ground control points, one or moreembodiments of the image georeferencing system can apply atransformation that modifies the point cloud, such that each point inthe point cloud is defined by coordinates relative to a location onEarth (e.g., distance and/or direction from a section corner or othersurvey point).

Turning now to FIG. 1, additional detail will be provided regardingcomponents and capabilities of one or more embodiments of the imagegeoreferencing system. In particular, FIG. 1 shows a schematic diagramillustrating an example embodiment of an image georeferencing system(“georeferencing system”) 100. As shown in FIG. 1, in one or moreembodiments, the georeferencing system 100 includes an image receiver102, a georeference information manager 104, a utilization module 118,and a device storage manager 120. As illustrated, the georeferenceinformation manager 104 may also include a feature detector 106, afeature matcher 108, a filter facility 110, a three-dimensionalrepresentation manager 112, a homography facility 114, and an orthomapmanager 116. Moreover, the device storage manager 120 may also include,as shown, image data 122 and georeference data 124.

Each of the components 102-120 of the georeferencing system 100 andtheir corresponding elements may be in communication with one anotherusing any suitable communication technologies. It will be recognizedthat although components 102-120 are shown to be separate in FIG. 1, anyof components 102-120 may be combined into fewer components, such asinto a single component, divided into more components, or configuredinto different components as may serve a particular embodiment.Moreover, one or more embodiments of the georeferencing system 100 mayinclude additional components or fewer components than those illustratedin FIG. 1.

The components 102-120 and their corresponding elements can comprisesoftware, hardware, or both. For example, the components 102-120 andtheir corresponding elements can comprise one or more instructionsstored on a computer-readable storage medium and executable byprocessors of one or more computing devices. When executed by the one ormore processors, the computer-executable instructions of thegeoreferencing system 100 can cause one or more computing systems (e.g.,one or more server devices) to perform the methods and provide thefunctionality described herein. Alternatively, the components 102-120can comprise hardware, such as a special purpose processing device toperform a certain function or group of functions. Additionally oralternatively, the components 102-120 can comprise a combination ofcomputer-executable instructions and hardware.

Furthermore, the components 102-120 of the georeferencing system 100 andtheir corresponding elements may, for example, be implemented as one ormore stand-alone applications, as one or more modules of an application,as one or more plug-ins, as one or more library functions or functionsthat may be called by other applications, and/or as a cloud-computingmodel. Thus, the components 102-120 of the georeferencing system 100 andtheir corresponding elements may be implemented as a stand-aloneapplication, such as a desktop or mobile application. Furthermore, thecomponents 102-120 of the georeferencing system 100 may be implementedas one or more web-based applications hosted on a remote server.Alternatively or additionally, the components of the georeferencingsystem 100 may be implemented in a suite of mobile device applicationsor “apps.”

As previously mentioned, and as illustrated in FIG. 1, thegeoreferencing system 100 may include the image receiver 102. In one ormore embodiments, the image receiver 102 receives, gathers, captures,accesses, provides, and/or identifies one or more images and relateddata for utilization by the georeferencing system 100. In particular,the image receiver 102 can access or capture one or more aerial images.In one or more embodiments, the image receiver 102 accesses one or moreaerial images previously captured by a UAV (e.g., and uploaded to thegeoreferencing system 100). Similarly, the image receiver 102 cancapture one or more aerial images from aboard a UAV.

The image receiver 102 can perform its functions with regard to avariety of aerial images. For example, the image receiver 102 canreceive, access, or capture color images, black and white images,gray-scale images, microwave images, infrared images, ultra violetimages, x-ray images, or gamma ray images. Similarly, the image receiver102 can receive, access, or capture images in a variety of formats,including, but not limited to, TIFF, JPEG, PNG, GIF, or other imagetypes. The georeferencing system 100 can utilize any images describedherein to perform its functions.

In addition to images, the image receiver 102 can also receive, gather,capture, access, provide, or identify data related to one or moreimages. For instance, the image receiver 102 can access or capture dateand time data associated with one or more images, georeferenceinformation associated with one or more images, or other image data. Inparticular, the image receiver 102 can capture images containing one ormore ground control points and access corresponding data related to thelocation of the one or more ground control points. Similarly, the imagereceiver 102 can capture GPS data associated with one or more images.For example, the image receiver 102 can obtain GPS data related to thelocation of a UAV at the time a camera on board the UAV captures one ormore images.

As shown in FIG. 1, the georeferencing system 100 may also include thegeoreference information manager 104. The georeference informationmanager 104 can access, create, calculate, generate, produce, provide,modify, and/or project georeference information. As further outlinedbelow with regard to its individual components, the georeferenceinformation manager 104 can generate georeference information withregard to one or more aerial images, orthomaps, three-dimensionalrepresentations, or point clouds. Moreover, the georeference informationmanager 104 can create georeference information utilizing any number oftools, algorithms, or methods.

For instance, in one or more embodiments, the georeference informationmanager 104 compares a first set of images containing ground controlpoints (e.g., images captured from image receiver 102) to a new set ofimages without ground control points. In particular, the georeferenceinformation manager 104 identifies key features (e.g., utilizing featuredetector 106) matches features from the first set of images to featuresfrom the new set of images (e.g., utilizing feature matcher 108),filters out outliers, erroneous matches, and/or undesired objects orfeatures (e.g., utilizing filter facility 110), and utilizes the matched(and/or filtered) features to generate georeference information withregard to the new set of images.

For example, the georeference information manager 104 can utilizematched features to generate georeference information in the form of ageoreferenced three-dimensional representation (e.g., a point cloud). Asdiscussed above, a point cloud may consist of three-dimensional pointscorresponding to the features of a set of images. In one or moreembodiments, the georeference information manager 104 generates a pointcloud (e.g., utilizing three-dimensional representation manager 112)based on the first set of images and the new set of images. Moreover,and as described in more detail below, the georeference informationmanager 104 applies a transformation to the point cloud to generate ageoreferenced point cloud from the first set of images and the new setof images.

Similarly, the georeference information manager 104 can utilize matchedfeatures to calculate a homography (e.g., utilizing the homographyfacility 114). In particular, the georeference information manager 104can generate a homography that relates points from a first aerial imageto a second aerial image. More specifically, the georeferenceinformation manager 104 can generate a homography that projects groundcontrol points (e.g., georeference information) from a first aerialimage to a second aerial image that did not originally show groundcontrol points.

Moreover, the georeference information manager 104 can utilize matchedfeatures to generate georeference information in the form of ageoreferenced orthomap. In particular, the georeference informationmanager 104 can generate a georeferenced orthomap (e.g., utilizing theorthomap manager 116) by identifying matched features from an originalorthomap showing ground control points and a new image (not showingground control points). Specifically, the georeference informationmanager 104 can generate a homography based on matched features betweenthe original orthomap and the new image and project features of the newimage onto the original orthomap to generate a new georeferencedorthomap. Additional detail will be provided regarding the specificfunctions of the georeference information manager 104 with regard to itsindividual components, as follows.

As just discussed, and as illustrated in FIG. 1, the georeferenceinformation manager 104 may include the feature detector 106. Thefeature detector 106 can detect and identify features in an image. Inparticular, the feature detector 106 can identify distinct features orkey points in one or more aerial images. For example, the featuredetector 106 can identify edges, corners, regions, structures,vegetation, or other objects (or portions of objects) within an image.

Particularly, the feature detector 106 can identify one or more groundcontrol points within an image. For instance, in one or moreembodiments, the feature detector 106 can automatically identify groundcontrol points in an image based on one or more patterns. For example,the feature detector 106 can compare characteristics of a ground controlpoint shown in an image (e.g., the shape, color, size, etc.) to one ormore pre-defined patterns. Based on the comparison, the feature detector106 can determine that a portion of an image comprises a ground controlpoint. More specifically, the feature detector can identify a portion ofan image containing a white “X,” compare the white “X” to a pre-definedpattern (e.g., a white “X” pattern) and determine that the white “X”shown in an image is a ground control point.

Additionally, or alternatively, the feature detector 106 can identifyground control points based on user input. For instance, in one or moreembodiments, the feature detector 106 can receive user input indicatingthat one or more points in a point cloud represent a ground controlpoint. Similarly, in one or more embodiments, the feature detector 106can receive user input identifying ground control points shown on anorthomap. User input in this context can include user input identifyingthe ground control points (e.g., selecting points representative ofground control points) and/or user input defining the location of theground control points (e.g., user input of the coordinates of the groundcontrol points).

The feature detector 106 may utilize a variety of algorithms to identifyfeatures. For example, one or more embodiments utilize a scale-invariantfeature transform (or SIFT) to identify features. Other embodiments mayutilize a speeded up robust features (SURF) algorithm, AKAZE, or anotherfeature detector algorithm. Additionally or alternatively, the featuredetector 106 may rely on user input to identify features.

Moreover, as shown in FIG. 1, the georeference information manager 104includes the feature matcher 108. The feature matcher 108 identifiescorresponding features between or among images, orthomaps, orthree-dimensional representations. For instance, the feature matcher 108may compare features identified by the feature detector 106 fromdifferent images and determine what features match from the differentimages. For example, the feature matcher 108 may compare the edge of abuilding in a first image and the edge of a building in a second imageto determine whether the edge in the first image corresponds to the edgein the second image.

The feature matcher 108 may utilize a variety of algorithms to identifycommon features. For example, the feature matcher 108 may utilizeclassical cross-correlation feature matching, variance normalizedcross-correlation feature matching, color-based mean-square-errorfeature matching, iterative closest point feature matching, or a SIFTalgorithm. In one or more embodiments, the feature matcher 108 utilizesuser input to identify one or more common features.

In addition, as illustrated in FIG. 1, the georeference informationmanager 104 may also include the filter facility 110. In one or moreembodiments, the filter facility 110 manages one or more filters andapplies the one or more filters to one or more images orthree-dimensional representations. For instance, the filter facility 110can apply filters to assist in identifying common features between oramong images, orthomaps, or three-dimensional representations. Moreover,the filter facility 110 can identify and filter out outliers and limitfalse matches. More specifically, in one or more embodiments, the filterfacility 110 utilizes a random sample consensus (RANSAC) algorithm toidentify outliers. The filter facility 110 may operate as a separatecomponent of the georeferencing system 100 or it may operate as part ofother components; for example, the three-dimensional representationmanager 112, the feature matcher 108, the homography facility 114,and/or the orthomap manager 116 may communicate with the filter facility110, or incorporate the functions of the filter facility 110 into theirown operation.

For example, in comparing features between images captured on differentdates, the georeferencing system 100 may erroneously determine that twodifferent features shown in two images are actually a common featureshown in the two images. The filter facility 110 can apply one or morefilters to identify and remove outliers so as to avoid skewinginformation generated by the georeferencing system 100. In this way, forexample, the georeferencing system 100 can robustly estimatethree-dimensional representations, orthomaps, and homographies despitethe fact that inaccuracies (e.g., false matches) may initially appear inthe data.

In addition to filtering features between images, the filter facility110 can apply a filter to other types of data. For instance, asdiscussed in greater detail below, the filter facility 110 can identifyand remove outliers with regard to observed positions of one or moreground control points. The filter facility 110 can identify outliers(e.g., utilizing RANSAC) in any data collected, utilized, or generatedby the georeferencing system 100.

The filter facility 110 can modify various parameters of filtersemployed by the filter facility 110. For example, the filter facility110 can modify parameters so that filters will identify additional, orfewer, outliers. For example, in comparing an image of a site on a firstday with an image of the site on a different day, the images may containa number of features that are similar, but slightly different. In such acircumstance, the filter facility 110 can modify parameters so that theapplicable filter will be more sensitive to outliers (and thus, forexample, avoid false matches).

The filter facility 110 can modify the parameters of one or more filtersbased on a variety of factors. In one or more embodiments, the filterfacility 110 modifies filter parameters based on user input (e.g., auser defines modified filter parameters). In other embodiments, thefilter facility 110 modifies filter parameters based on the date and/ortime of captured images, the number of corresponding features betweenimages, or some other factor. For example, if the georeferencing system100 captures a first set of images a week apart from a second set ofimages, the filter facility 110 can modify filter parameters based onthe relative difference in the time of capture between the two set ofimages.

The filter facility 110 can apply a number of different parameters inidentifying outliers. For example, in comparing images the filterfacility 110 may identify outliers based on geometry. Thus, for example,the filter facility 110 may require geometric consistency betweenfeatures before permitting the feature matcher 108 to identify a commonfeature between two images. Similarly, the filter facility 110 mayidentify outliers based on color, orientation, pattern, position (e.g.,position relative to another feature), or other factors.

In one or more embodiments, the filter facility 110 removes one or morefeatures or objects from one or more images or three-dimensionalrepresentations (e.g., features identified and/or matched utilizing thefeature detector 106 and the feature matcher 108). The filter facility110 can remove such features or objects. For example, the filterfacility 110 may remove temporary or mobile objects, such as vehicles,bikes, individuals, animals, or construction equipment. Similarly, thefilter facility 110 may remove other undesired items, such asvegetation.

The filter facility 110 may also filter or modify other features. Forexample, in one or more embodiments the filter facility 110 can accountfor perspective effects, camera effects, or other effects. Similarly thefilter facility 110 can modify one or more images such that the imagescan be approximated to a plane. For example, the filter facility 110 canmodify an image with perspective effect so that the image isapproximated to a plane for comparison to another image approximated toa plane. For instance, the filter facility 110 can generate anorthographic projection of a mesh obtained from a scene reconstructionalgorithm (e.g. Poisson Reconstruction) where camera parameterscorresponding to that image where found and, consequently, integratedinto the output mesh.

The filter facility 110 can remove objects based on a number of factors.For example, the filter facility 110 can remove objects based on userinput, shape, color, pattern, texture, comparison between one or moreimages, comparison between one or more features, or other factors. Inparticular, the filter facility 110 can compare an object in an imagewith a set of pre-defined patterns (e.g., shape of a vehicle) and removethe object based on a correspondence between the object in the image andone of the set of predefined patterns. Similarly, the filter facility110 can remove an object based on the fact that the object appears atone location in a first image (or orthomap or three-dimensionalrepresentation) and appears at a second location in a second image(i.e., indicating that the object is mobile and/or temporary).Alternatively or additionally, the georeferencing system 100 canidentify objects to remove based on user input. For example, a user canselect (through user input provided via a computing device) objects(e.g., an automobile, vegetation, etc.) represented in an aerial image,orthomap, or three-dimensional representation that the filter facility110 should remove.

In addition to the filter facility 110, as shown in FIG. 1, thegeoreference information manager 104 may also include thethree-dimensional representation manager 112. The three-dimensionalrepresentation manager 112 can generate, calculate, create, modify,produce, receive, provide, transform, reference, and/or utilize one ormore three-dimensional representations. In particular, thethree-dimensional representation manager 112 can communicate with theimage receiver 102 to obtain one or more images and create one or morethree-dimensional representations from the one or more images.Similarly, the three-dimensional representation manager 112 may interactwith the feature detector 106, the feature matcher 108, and the filterfacility 110, to identify features, compare images or three-dimensionalrepresentations, and generate or modify three-dimensionalrepresentations.

The three-dimensional representation manager 112 can generate one ormore unreferenced three-dimensional representations in a number of ways.In particular, the three-dimensional representation manager 112 cancreate an unreferenced three-dimensional representation based on one ormore images of a site. For instance, the three-dimensionalrepresentation manager 112 can utilize one or more algorithms to converttwo-dimensional images into an unreferenced three-dimensionalrepresentation. For example, in one or more embodiments thethree-dimensional representation manager 112 utilizes a structure frommotion algorithm (e.g., applies a sequential or iterative structure frommotion algorithm utilizing a plurality of images) to generate athree-dimensional point cloud from a set of two-dimensional images witha certain degree of overlap.

The three-dimensional representation manager 112 can also generategeoreferenced three-dimensional representations. For example, in one ormore embodiments the three-dimensional representation manager 112 canutilize georeference information related to one or more images togenerate a georeferenced point cloud. In particular, thethree-dimensional representation manager 112 can identify ground controlpoints shown in one or more images and, based on the ground controlpoints and the one or more images, generate a georeferenced point cloud,as explained in more detail below. Alternatively or additionally, thethree-dimensional representation manager 112 can obtain GPS informationrelated to one or more images (e.g., from the image receiver 102 orgeoreference data 124) and utilize the global position information togenerate a georeferenced point cloud.

As mentioned previously, in some circumstances the process of generatinggeoreferenced three-dimensional representations (e.g., sequentialstructure from motion processes) can cause error accumulation, or drift.Accordingly, in one or more embodiments, the three-dimensionalrepresentation manager 112 can modify one or more three-dimensionalrepresentations to correct for such errors. In particular, one or moreembodiments of the three-dimensional representation manager 112 refinesground control points and utilizes an additional constrained bundleadjustment algorithm based on the refined ground control points tocorrect for drift. In other words, the three-dimensional representationmanager 112 can utilize refined ground control points as anchors tocorrect errors in a point cloud via a constrained bundle adjustmentalgorithm.

One or more embodiments of the three-dimensional representation manager112 generates refined ground control points by comparing the observedthree-dimensional positions of ground control points to the measuredthree-dimensional positions of the ground control points. For instance,as described in greater detail below, in one or more embodiments thethree-dimensional representation manager 112 generates refined groundcontrol points by calculating the observed positions of one or moreground control points from a plurality of aerial images; filtering theobserved positions; and comparing the filtered, observed positions tothe real-world, measured position of the ground control points.

Moreover, in one or more embodiments, the georeferencing systemgenerates refined ground control points relative to the coordinatesystem of a point cloud. Accordingly, in one or more embodiments thethree-dimensional representation manager 112 generates refined groundcontrol points by further calculating a transformation. In particular,the three-dimensional representation manager 112 can calculate atransformation between filtered, observed positions of ground controlpoints and measured positions of the ground control points. In addition,the three-dimensional representation manager 112 can apply thetransformation inversely to convert the measured positions into therelative coordinate system of the point cloud. In this manner, thethree-dimensional representation manager 112 can calculate refinedground control points relative to the point cloud (e.g., in terms of thecoordinate system corresponding to the point cloud).

Moreover, in one or more embodiments, the three-dimensionalrepresentation manager 112 utilizes the refined ground control points toaccount for drift in a three-dimensional representation. For example, insome embodiments the three-dimensional representation manager 112applies a constrained bundle adjustment algorithm to a three-dimensionalrepresentation (e.g., a three-dimensional point cloud) utilizing therefined ground control points. Specifically, in one or more embodiments,the three-dimensional representation manager 112 applies a bundleadjustment algorithm while setting the refined ground control points asconstant, thus, holding the refined ground control points in place whileapplying the bundle adjustment algorithm to the three-dimensionalrepresentation. Similarly, one or more embodiments of thethree-dimensional representation manager 112 apply a constrained bundleadjustment algorithm that weights the refined ground control points(e.g., applies a scaling factor to a cost or error value associated withthe refined ground control points). By applying a constrained bundleadjustment algorithm in this manner, in one or more embodiments thethree-dimensional representation manager 112 generates a correctedthree-dimensional representation that corrects drift based on themeasured position of ground control points.

In addition to generating point clouds that correct for accumulationerrors, the three-dimensional representation manager 112 can alsogenerate a georeferenced point cloud from an unreferenced point cloud.For instance, the three-dimensional representation manager 112 canidentify one or more data points within an unreferenced point cloud,identify corresponding georeference information, and generate ageoreferenced point cloud based on the georeference information. Morespecifically, in one or more embodiments, the three-dimensionalrepresentation manager 112 can identify one or more data points withinan unreferenced point cloud that represent ground control points, and,based on the known location on Earth of the ground control points,convert the unreferenced point cloud to a georeferenced point cloud.

To generate a georeferenced point cloud from an unreferenced pointcloud, in one or more embodiments the three-dimensional representationmanager 112 utilizes a transformation. As used herein, the term“transformation” refers to an algorithm that modifies data within athree-dimensional space based on reference points (e.g., ground controlpoints or GPS data). In particular, a transformation can modifyrotation, scaling, and translation of data points within an unreferencedpoint cloud to relate the data points to corresponding geographiclocations, thereby creating georeferenced data points. In other words,the transformation can modify an unreferenced point cloud based on knownground control points to generate a georeferenced point cloud, in whicheach of the points in the point cloud are defined in relation to alocation on Earth. In some instances, the three-dimensionalrepresentation manager 112 can utilize a Helmert transformationalgorithm to transform an unreferenced point cloud into a georeferencedpoint cloud.

In one or more embodiments, the three-dimensional representation manager112 can also access a transformation generated from a first point cloudand apply the transformation to a second point cloud. For example, thethree-dimensional representation manager 112 can apply a previouslygenerated transformation to a new unreferenced point cloud to create ageoreferenced point cloud. For instance, as discussed above, thethree-dimensional representation manager 112 can generate an initialpoint cloud based on a first set of images of a site and calculate aninitial transformation based on ground control points reflected in thefirst set of images. Thereafter, the three-dimensional representationmanager 112 can access a new set of images of the site and generate anew point cloud based, at least in part, on the new set of images.Because the three-dimensional representation manager 112 had alreadycalculated a transformation in relation to the first set of images (and,in some examples, data from the first set of images provides structureto the new point cloud), the three-dimensional representation manager112 can apply the initial transformation to generate a new georeferencedpoint cloud. In some examples, this process may include updating theinitial point cloud with data from the new set of images and applyingthe transformation to the data from the new set of images.

Aside from generating a georeferenced three-dimensional representationfrom data representative of aerial images showing known ground controlpoints, the three-dimensional representation manager 112 can furthermodify a three-dimensional representation. For example, in one or moreembodiments the three-dimensional representation manager 112 modifies anexisting three-dimensional representation generated from a first set ofimages based on one or more new images. In particular, thethree-dimensional representation manager 112 can utilize one or more newimages to update a three-dimensional representation generated from afirst set of images. For example, the three-dimensional representationmanager 112 can obtain a set of aerial images for a site captured on afirst day, generate a point cloud based on the set of aerial images,obtain a new set of images for the site captured on a second day, andmodify the point cloud based on the new set of images.

Additionally or alternatively, the three-dimensional representationmanager 112 can also generate a single point cloud based on two sets ofaerial images captured at different times. Thus, the three-dimensionalrepresentation manager 112 can access a set of aerial images for a sitecaptured on a first day, access a new set of images for the sitecaptured on a second day, and generate a point cloud based on both setsof aerial images.

In generating or modifying a three-dimensional representation, thethree-dimensional representation manager 112 can prioritize one or morefeatures. In particular, the three-dimensional representation manager112 can emphasize the features of one image or the features of anotherimage in generating a three-dimensional representation. For example, thethree-dimensional representation manager 112 can prioritize one or morefeatures based on the time an image was captured, the characteristics ofa feature within an image, a comparison of features contained within aset of images, a confidence factor associated with one or more features,user input, and/or any other suitable factors.

In particular, the three-dimensional representation manager 112 mayprioritize features of a later-captured image over the features of anearlier-captured image. For instance, a user may seek to generate ageoreferenced three-dimensional representation with regard to thefeatures of a site on a second day based on images captured on thesecond day, while only having access to georeference information withregard to images captured at the site on the first day. In one or moreembodiments, the three-dimensional representation manager 112 can builda georeferenced point cloud based on both the images captured on thefirst day and the images captured on the second day, while prioritizingthe features of the images captured on the second day.

Thus, for example, if images on the first day show a structure that isno longer shown on images from the second day, the three-dimensionalrepresentation manager 112 can generate a three-dimensionalrepresentation that does not reflect the removed structure. Similarly,if images on the first day do not show a structure, but images from thesecond day do show a new structure, the three-dimensional representationmanager 112 can generate a three-dimensional representation thatreflects the added structure. In addition, where images from the firstday show a structure, and images from the second day show a modifiedstructure, the three-dimensional representation manager 112 can generatea three-dimensional representation that reflects the modified structure.Where images from the first day and images from the second day show thesame structure, the three-dimensional representation manager 112 canutilize both sets of images to generate a georeferencedthree-dimensional representation.

Aside from the time of capture, the three-dimensional representationmanager 112 may prioritize the features of one or more images based onuser input. For instance, a user may provide user input identifying aparticular set of images, and the three-dimensional representationmanager 112 may prioritize the features in the identified set of images.Similarly, a user may select particular features from a particular setof images, and the three-dimensional representation manager 112 canprioritize those particular features in generating a three-dimensionalrepresentation.

As discussed above, the georeferencing system 100 can filter one or morefeatures of an image or three-dimensional representation. Accordingly,in prioritizing features (and performing its other functions), thethree-dimensional representation manager 112 and the georeferencingsystem 100 can filter features. For example, the three-dimensionalrepresentation manager 112 (e.g., in conjunction with the featurematcher 108 and the filter facility 110) can identify features appearingin one or more images and filter the images so that certain identifiedfeatures do not appear in a three-dimensional representation.

In addition to the three-dimensional representation manager 112, asillustrated in FIG. 1, the georeference information manager 104 may alsoinclude the homography facility 114. The homography facility 114 cancalculate, generate, estimate or create one or more homographies. Asused herein, the term “homography” refers to a mapping from points inone view to points in another view. In particular, a homography cancomprise a matrix, equation, or other mathematical construct relating adata point on a plane represented in a first image to the same point inthe plane represented in a second image. Thus, for example, a homographycan comprise a matrix that relates a pair of planar surfaces as viewedin multiple images. Stated differently, images of the same planarsurface in space are related by a homography.

The homography facility 114 can calculate a homography relating datapoints from different aerial images. For example, the homographyfacility 114 can calculate a homography relating data points in a firstimage (e.g., an aerial image or an orthomap) to a data point in a secondimage (e.g., another aerial image or another orthomap). In one or moreembodiments, the homography facility 114 generates a homography byidentifying common features (e.g., from the feature detector 106 and thefeature matcher 108) in a plurality of images and generating thehomography based on the location of the common features in the pluralityof images.

As mentioned, the homography facility 114 can calculate a homographycapable of relating points in a first image to points in a second image.Accordingly, the homography facility 114 can calculate a homographycapable of relating a ground control point shown in a first image to alocation on a second image, in which the ground control point is notshown. More particularly, the homography facility 114 can calculate thelocation of ground control points in reference to an image capturedwithout ground control points based on an image showing ground controlpoints.

The georeferencing system 100 and the homography facility 114 canaccount for outliers or inaccuracies in data (e.g., by utilizing thefilter facility 110). For example, the homography facility 114 canaccount for inaccuracies in calculating a homography utilizing commonfeatures between images. For example, in one or more embodiments thegeoreferencing system 100 utilizes a RANSAC algorithm to estimate ahomography while accounting for outliers in the data. Thus, thehomography facility 114 can robustly estimate a homography that is notunduly affected by outliers or inaccurate feature matches.

Furthermore, as illustrated in FIG. 1, the georeference informationmanager 104 may also include the orthomap manager 116. The orthomapmanager 116 can receive, generate, create, modify, or provide one ormore orthomaps. In particular, the orthomap manager 116 can generate ageoreferenced orthomap. For example, the orthomap manager 116 cangenerate a georeferenced orthomap for one or more images capturedwithout georeference information.

In one or more embodiments, the orthomap manager 116 can utilize ahomography (e.g., a homography generated by the homography facility 114)to generate a georeferenced orthomap. For example, in one or moreembodiments, the orthomap manager 116 can utilize a homography relatingpoints on a georeferenced orthomap and points on a new image to generatea georeferenced orthomap showing features of the new image. Thus, theorthomap manager 116 can project a new image onto an orthomap.

Moreover, the orthomap manager 116 can generate an orthomap based on athree-dimensional representation. In particular, the orthomap manager116 can generate a georeferenced orthomap utilizing a georeferencedpoint cloud. For example, the orthomap manager 116 can receive ageoreferenced point cloud from the three-dimensional representationmanager 112 and convert the georeferenced point cloud to a georeferencedorthomap.

As discussed above with regard to three-dimensional representationmanager 112, in one or more embodiments the orthomap manager 116 canalso prioritize one or more features. For instance, the orthomap manager116 can prioritize the features of a plurality of new images in relationto the features of a plurality of other images. As mentioned previouslythe georeferencing system 100 and the orthomap manager 116 canprioritize features based on the time an image was captured, thecharacteristics of a feature within an image, a comparison of featurescontained within a set of images, user input, or some other feature.

In one or more embodiments, the orthomap manager 116 can modify one ormore orthomaps. For example, the orthomap manager 116 can modify anorthomap based on a new image. In particular, the orthomap manager 116can modify an orthomap to reflect features shown in a new image.

Additionally, the orthomap manager 116 can modify the appearance of anorthomap. For example, the orthomap manager 116 can modify the tone,hue, brightness, color, shade, or other visual characteristics of anorthomap. For example, if the orthomap manager 116 modifies or generatesan orthomap based on a new image, the orthomap manager 116 can modifythe visual characteristics of the orthomap based on the visualcharacteristics of the new image. Thus, the orthomap manager 116 canblend the new image and the orthomap. Alternatively or additionally, theorthomap manager 116 can preserve the visual characteristics of anorthomap and modify the visual characteristics of a new image to matchthe orthomap.

As illustrated in FIG. 1, the georeferencing system 100 may also includethe utilization module 118. In one or more embodiments the utilizationmodule 118 can provide data generated by the georeferencing system 100to one or more users or devices. In particular, the georeferencingsystem 100 can provide, for display, images or georeference informationto one or more users. For example, the utilization module 118 canprovide for display to a user a representation or model of ageoreferenced three-dimensional representation generated by thegeoreferencing system 100. Similarly, the utilization module 118 canprovide for display a georeferenced orthomap or a georeferenced aerialimage. Additionally or alternatively, the utilization module 118 canprovide for display a three-dimensional representation, orthomap, oraerial image that identifies real-world location information generatedby the georeferencing system 100.

The utilization module 118 can also provide for display images orgeoreference information in relation to other images, designs, plans, orgraphics. For example, the utilization module 118 can provide fordisplay a georeferenced orthomap generated by the georeferencing system100 together with design plans. Specifically, the utilization module 118can provide a georeferenced orthomap positioned so that the features ofthe georeferenced orthomap and design plans align according to theirexisting (or planned) location on Earth.

Similarly, the utilization module 118 can provide for display acomparison between or among one or more three-dimensionalrepresentations, orthomaps, and/or images. For example, the utilizationmodule 118 can provide for display a georeferenced orthomap displayingthe differences between a first georeferenced orthomap reflecting a siteat a first point in time and a second georeferenced orthomap reflectinga site at a second point in time.

The utilization module 118 can also provide for display one or morereports, tables, charts, or summaries. For example, the utilizationmodule 118 can provide a summary regarding the percentage change of asite as reflected by a comparison between images, orthomaps, and/orthree-dimensional representations. Additionally or alternatively, theutilization module 118 can provide a report regarding the location offeatures, georeference information, or other information.

The utilization module 118 can also utilize images and/or georeferenceinformation. For example, the utilization module 118 can utilize imagesand or georeference information for navigation. Specifically, theutilization module 118 can utilize a georeferenced orthomap generated bythe georeferencing system 100 to navigate a UAV. For instance, theutilization module 118 can provide a georeferenced orthomap for displayto an individual navigating a UAN or the georeferencing system 100 canautonomously navigate the UAV utilizing a georeferenced orthomap.

As shown in FIG. 1, the georeferencing system 100 may also include thedevice storage manager 120. The device storage manager 120 maintainsdata for the georeferencing system 100. The device storage manager 120can maintain data of any type, size, or kind, as necessary to performthe functions of the georeferencing system 100.

As illustrated, the device storage manager 120 may include image data122. Image data 122 can include data representative of images captured,generated, received, modified, or otherwise utilized by thegeoreferencing system 100, including aerial images, orthomaps,three-dimensional representations, or other images. The image data 122may additionally or alternatively include data associated with one ormore images, such as metadata associated with one or more images. Moreparticularly, image data 122 may include georeference information storedas metadata with regard to one or more images.

Furthermore, as illustrated, the device storage manager 120 may alsoinclude georeference data 124. Georeference data 124 may include datarepresentative of georeference information or any other informationregarding a location on Earth. In particular, georeference data 124 mayinclude data relating features appearing within one or more images tocorresponding locations on Earth. For instance, georeference data 124can include data relating one or more structures or other features of ajob-site to corresponding geographic coordinates representative of thegeographic locations of the structures or other features on the earth'ssurface. Thus, for example, the georeference data 124 can include GPSdata, geospatial data, or any other data that relates image data tocorresponding geographic locations.

Turning now to FIG. 2, further information will be provided regardingimplementation of the georeferencing system 100. Specifically, FIG. 2illustrates a schematic diagram of one embodiment of an exemplary systemenvironment (“system”) 200 in which the georeferencing system 100 canoperate. As illustrated in FIG. 2, the system 200 can include clientdevices 202 a-202 b, a network 204, a server 206, a UAV 208, and aground station 210. The client devices 202 a-202 b, the network 204, theserver 206, the UAV 208, and the ground station 210 may becommunicatively coupled, as shown in FIG. 2. Although FIG. 2 illustratesa particular arrangement of the client devices 202 a-202 b, the network204, the server 206, the UAV 208, and the ground station 210, variousadditional arrangements are possible. For example, the client devices202 a-202 b (or the UAV 206 and/or ground station 210) may directlycommunicate with the server 206, bypassing the network 206. Similarly,in one or more embodiments the client devices 202 a-202 b may directlycommunicate with each other.

In yet other embodiments, the client 202 b may communicate directly withthe UAV 208, which may communicate directly with the ground station 210via the local connection 212. The local connection 212 may comprise anyrecognized form of wired or wireless communication. For example, in oneor more embodiments the client device 202 b may include a mobilecomputing device (e.g., tablet) utilized by a UAV operator tocommunicate with the UAV 208 and the ground station 210. Morespecifically, a UAV operator may utilize client the device 202 b to sendinformation to, and receive information from, the UAV 208; the UAV 208may dock with (and communicate with) the ground station 210; and theclient device 202 b, the UAV 208, and/or the ground station 210 canprovide information to the server 206.

The georeferencing system 100 may be implemented in whole, or in part,by the individual elements 202 a-210 of the system 200. For instance,the georeferencing system 100 may be implemented on client device 202 a(e.g., the georeferencing system 100 may correspond to client device 202a). Similarly, the georeferencing system 100 may be implemented on theserver 206. Alternatively or additionally, different components andfunctions of the georeferencing system 100 may be implemented separatelyamong client devices 202 a-202 b, the server 206, the UAV 208, and theground station 210. For instance, the image receiver 102 may beimplemented as part of the client device 202 c, the three-dimensionalrepresentation manager 112 may be implemented as part of the server 206,the utilization module 118 may be implemented as part of client device202 a, and the device storage manager 120 may be implemented as part ofboth client device 202 a and the server 206.

The client devices 202 a-202 b, the network 204, the server 206, the UAV208, and the ground station 210 may communicate using any communicationplatforms and technologies suitable for transporting data and/orcommunication signals, including any known communication technologies,devices, media, and protocols supportive of remote data communications,examples of which will be described in more detail below with respect toFIG. 11.

The client devices 202 a-202 b may comprise any computing device. Forexample, client devices 202 a-202 b may comprise one or more personalcomputers, laptop computers, mobile devices, mobile phones, tablets,special purpose computers, TVs, or other computing devices. Similarly,client devices 202 a-202 b may also comprise one or more UAVs, groundstations, or other computing devices.

As used herein, the term “UAV” or “unmanned aerial vehicle” refers to anaircraft that can be piloted autonomously or remotely by a controlsystem. The UAV 208 may comprise any type of UAV, including micro UAV,low altitude UAV, or high altitude UAV, whether autonomously or remotelypiloted. Similarly, the UAV 208 may include multi-rotor UAVs,single-rotor UAVs, blimp UAVs, or other types of UAVs. In particular,the UAV 208 may include an onboard computer that controls the autonomousflight of the UAV. In at least one embodiment, the UAV 208 is amulti-rotor vehicle, such as a quadcopter, and includes a carbon fibershell, integrated electronics, a battery bay, a global positioningsystem (“GPS”) receiver, a fixed or swappable imaging system (e.g., adigital camera), and various additional sensors and/or receivers. TheUAV 208 may contain one or more computer-readable storage media and/orone or more processors with instructions stored thereon that, whenexecuted by the one or more processors cause the UAV 208 to performfunctions described herein.

Alternatively or additionally, the georeferencing system 100 may beimplemented as part of ground station 210. The ground station 210 may beutilized to land, store, charge, or repair the UAV 208. Moreover, theground station 210 may be utilized to communicate with the UAV 208 priorto, during, or after a flight.

Although FIG. 2 illustrates two client devices 208 a-208 b, the singleUAV 208, and the single ground station 210, it will be appreciated thatthe client devices 202 a-202 b, the UAV 208, and the ground station 210can represent any number of computing devices, UAVs, or ground stations(fewer or greater than shown). Moreover, client devices 202 a-202 b maycomprise any client device (e.g., computing device) described herein,including those described with regard to FIG. 11.

In addition, and as illustrated in FIG. 2, the client devices 202 a-202b, the server 206, the UAV 208, and the ground station 210 maycommunicate via the network 204. For example, the UAV 208 maycommunicate with the client device 202 b, ground station 210, or theserver 206 directly or via the network 204. Further, in one or moreembodiments the server, the client device 202 b, the ground station 210,or the server 206 can provide navigation information to the UAV 208.More particularly, the client device 202 b, the ground station 210, orthe server 206 may provide navigation information to the UAV 208 forpurposes of navigating, taking off, and/or landing. Similarly, in one ormore embodiments the UAV 208 may provide information to the clientdevice 202 b, the ground station 210, or the server 206 during or afterflight. By way of further example, in at least one embodiment the UAV208 may provide image data or georeference information to the clientdevice 202 b, the ground station 210, or the server 206. Further, theserver 206 can provide images, communications, summaries, reports, orother data to device 202 a (e.g., a personal computer operated by auser).

The network 204 may represent a network or collection of networks (suchas the Internet, a corporate intranet, a virtual private network (VPN),a local area network (LAN), a wireless local network (WLAN), a cellularnetwork, a wide area network (WAN), a metropolitan area network (MAN),or a combination of two or more such networks. Thus, the network 204 maybe any suitable network over which the client device 202 a may accessthe server 206 and/or the client devices 202 a-202 d, or vice versa. Thenetwork 204 will be discussed in more detail below with regard to FIG.11.

The server 206 may generate, store, receive, and transmit any type ofdata, including image data, georeference information, navigationinformation, or electronic communication data. For example, the server206 may receive data from a client device, such as the client device 202a, and send the data to another client device, such as the client device202 b, 202 c, and/or 202 d. The server 206 can also transmit electronicmessages between one or more users of the system 200. In one exampleembodiment, the server 206 is a data server. The server 206 can alsocomprise a communication server or a web-hosting server. Additionaldetails regarding the server 206 will be discussed below with respect toFIG. 11.

In addition to the shown elements of system 200, one or more users maybe associated with the client devices 202 a-202 d. Users may beindividuals (i.e., human users). A user may interact with client devices202 a-202 d by way of a user interface provided via a display deviceassociated with client devices 202 a-202 d. For instance, a user caninteract with the computing device 202 a for the purpose of reviewingimages or information provided by the georeferencing system 100 and/ordeciding policies based on that information. Similarly, a user canprovide user input via the client device 202 a for operation of thegeoreferencing system 100. For example, one or more users can provideinput of parameters, identify pertinent images to utilize, provide userinput of commands to generate a three-dimensional representation,orthomap, or georeference information, etc.

To illustrate, in one or more embodiments, a UAV operator may seek toutilize the UAV 208 to capture aerial images of a site. The UAV operatormay utilize client device 202 b (e.g., a tablet or other mobilecomputing device) to send instructions to the UAV 208. The UAV 208 mayfly from ground station 210 and capture images of the site. The UAV 208may then provide image data to the ground station 210 and/or clientdevice 202 b. The client device 202 b and/or the ground station 210 mayconvey the image data to the server 206 via the network 204.Furthermore, the UAV operator may obtain coordinates of ground controlpoints, and send the ground control points from the client device 202 bto the server 206 via the network 204. Thereafter, the server 206 maygenerate a georeferenced three-dimensional representation based on theimage data and the coordinates of ground control points and send thegeoreferenced three-dimensional representation to client device 202 a(e.g., a personal computer utilized by a designer).

At a later date, the UAV operator may seek to utilize the UAV 208 tocapture new aerial images of the site (without ground control points).The UAV operator may again utilize the client device 202 b, the UAV 208,and the ground station 210 to capture new image data and convey theimage data to the server 206. The server 206 may generate a newgeoreferenced three-dimensional representation utilizing the new imagedata and the previously captured image data. The server 206 may thentransmit the new georeferenced three-dimensional representation to theclient device 202 a.

Turning now to FIGS. 3A-3F, additional detail will be provided regardinggenerating georeference information utilizing the georeferencing system100. In particular, FIGS. 3A-3F illustrate an exemplary embodiment forgenerating a georeferenced three-dimensional representation inaccordance with one or more embodiments of the georeferencing system100.

FIG. 3A illustrates a representation of a plurality of images 300captured or accessed by the georeferencing system 100. As illustrated,the plurality of images 300 show a first structure 302, a secondstructure 304, a road 306, and ground control points 308 a, 308 b, 308c, and 308 d. Although FIG. 3A shows the first structure 302, the secondstructure 304, the road 306, and the ground control points 308 a-308 d,not every image in the plurality of images 300 need show each of theseelements. For example, one image may only show one ground control point308 a and the road 306; another image may only show the first structure302; and another image may show the second structure 304 and the groundcontrol point 308 d.

The georeferencing system 100 can capture the plurality of images 300 atvarying times. For example, the georeferencing system 100 can capturethe plurality of images 300 in a span of less than 15 minutes, less thanan hour, or less than a day. Alternatively or additionally, thegeoreferencing system 100 may capture the plurality of images 300 in aspan of multiple days, multiple weeks, multiple months, or even multipleyears.

The ground control points 308 a-308 d represent known points on Earth(e.g., known points within a geographic coordinate system). Inparticular, the ground control points 308 a-308 d can representtemporary markers placed prior to capturing the plurality of images 300.The georeferencing system 100 can identify the location of the groundcontrol points 308 a-308 d on Earth, for example based on a survey ofthe area and relation of the ground control points 308 a-308 d to knowngeographic coordinates (e.g. represented by survey monuments).

The georeferencing system 100 can obtain the location of the groundcontrol points at any point in time. For example, a surveyor can measurethe location of the ground control points 308 a-308 d on Earth prior toplacing the ground control points. Similarly, a surveyor can measure thelocation of the ground control points 308 a-308 d after placing theground control points, before the georeferencing system 100 captures theimages, or after the georeferencing system 100 captures the images.

FIG. 3B illustrates a representation of a three-dimensionalrepresentation in accordance with one or more embodiments. Asillustrated, the georeferencing system 100 converts the plurality ofimages 300 into a three-dimensional point cloud 310. The data points inthe point cloud 310 correspond to features of the plurality of images300. For example, as illustrated, data points in the point cloud 310correspond to the first structure 302, the second structure 304, theroad 306, and the ground control points 308 a-308 d. Moreover, asillustrated in FIG. 3B, the point cloud may also contain data pointsrepresenting other objects or features, for instance, pointsrepresenting the contours of the ground and other geographic features(e.g., terrain, hills, valleys, mountains, etc.).

As mentioned previously, a point cloud can comprise a set of data pointson a three-dimensional coordinate system. The georeferencing system 100can generate a three-dimensional point cloud from a set oftwo-dimensional images. For example, the georeferencing system 100 canidentify features in the plurality of images 300, and createthree-dimensional data points corresponding to the identified features.In particular, variations in how features are portrayed in the pluralityof images 300 (e.g., changes in perspective) permit the georeferencingsystem 302 to convert the features portrayed in the plurality of images300 (e.g., two-dimensional images) to three-dimensional points. Thegeoreferencing system 100 can combine three-dimensional pointscorresponding to features shown in the plurality of images 300 togenerate a three-dimensional point cloud.

In one or more embodiments, the georeferencing system 100 utilizes astructure from motion algorithm to convert two-dimensional images to athree-dimensional point cloud. In particular, in one or moreembodiments, the georeferencing system 100 compares the plurality ofimages 300, identifies features within the plurality of images 300, anddetermines matching features from the identified features. For example,with regard to FIG. 3A, the georeferencing system 100 can match an edgeof the first structure 302 shown in a first image in the plurality ofimages 300 with the edge of the first structure 302 shown in a secondimage in the plurality of images 300.

Moreover, in one or more embodiments, the georeferencing system 100tracks matching features from multiple images. For instance, thegeoreferencing system 100 can identify an edge of the second structure304 in a first image of the plurality of images 300, identify the edgeof the second structure 304 in a second image of the plurality of images300, and track variations in the edge of the second structure 304 fromimage to image.

Moreover, in at least one embodiment, the georeferencing system 100calculates feature trajectories by tracking matching features. Forexample, the georeferencing system 100 can calculate featuretrajectories by tracking the edge of the second structure 304 inmultiple images over time. Furthermore, in one or more embodiments thegeoreferencing system 100 utilizes the feature trajectories to constructa three-dimensional position of matching features.

Although the georeferencing system 100 can generate points within athree-dimensional point cloud based on two-dimensional images, in atleast one embodiment, the georeferencing system 100 can generate pointsbased on user input. For example, in one or more embodiments, the usercan input (or identify) the ground control points in the point cloud.

In one or more embodiments, the georeferencing system 100 can alsogenerate a georeferenced three-dimensional representation from GPS data.In particular, the georeferencing system 100 can estimate a cameraposition and latitude, longitude, and height utilizing GPS data. Thegeoreferencing system 100 can convert the camera position to a projectedreferential. The projected referential can then be utilized to generatea georeferenced three-dimensional representation from the plurality ofimages 300.

Moreover, in one or more embodiments, the georeferencing system 100accounts for variations in the plurality of images 300. For example, asdescribed previously, the georeferencing system 100 can account forvariations in perspective, calibration, and optical characteristics. Inparticular, the plurality of images 300 contains images captured atdifferent locations, angles, perspectives, illuminations, and so forth.One or more embodiments of the georeferencing utilize a bundleadjustment to account for variations in the plurality of images 300 andgenerate a three-dimensional point cloud. A bundle adjustment is analgorithm that refines a visual reconstruction to produce jointlyoptimal three-dimensional structure and viewing parameters. For example,in one or more embodiments, the georeferencing system 100 utilizes theLevenberg-Marquardt bundle adjustment method. Similarly, thegeoreferencing system 100 can utilize the gradient descent bundleadjustment method, the Newton-Rhapson bundle adjustment method, and/orthe Gauss-Newton bundle adjustment method.

As discussed previously, the georeferencing system 100 can generate atransformation and/or apply a transformation to a point cloud. FIG. 3Cillustrates a representation of applying a transformation to a pointcloud in accordance with one or more embodiments. In particular, FIG. 3Crepresents the georeferencing system 100 applying a transformation 320to the point cloud 310. The georeferencing system 100 utilizes theground control 308 a, 308 b, 308 c, and 308 d in the point cloud 310 andthe corresponding coordinate points 322 a, 322 b, 322 c, and 322 d togenerate and apply the transformation 320.

In particular, the coordinate points 322 a, 322 b, 322 c, and 322 drepresent the location on Earth of the ground control points 308 a, 308b, 308 c, and 308 d. For instance, the coordinate points 322 a-322 d maycomprise real-world coordinates of the ground control points 308 a-308d, respectively. As described above, for example, the coordinate points322 a-322 d may represent coordinates measured by a surveyor with regardto the location of temporary markers.

Utilizing the known coordinate points 322 a-322 d corresponding to theground control points 308 a-308 d, the georeferencing system 100 cancalculate and apply the transformation 320 to rotate, scale, andtranslate the point cloud 310. In particular, as shown, thegeoreferencing system 100 calculates and applies the transformation 320such that ground control points 308 a-308 d align with the coordinatepoints 322 a-322 d. Moreover, the georeferencing system 100 applies thetransformation 320 such that the remaining points in the point cloud 310align with the location on Earth corresponding to each point. As such,the georeferencing system 100 calculates and applies the transformation320 to generate a georeferenced point cloud.

For example, in constructing the point cloud 310, the georeferencingsystem 100 may build points in the point cloud 310 for the groundcontrol points 308 a-308 d at the following coordinates, respectively:(0,0,50), (0,10,50), (10,10,55), and (10, 0, 45). A surveyor may measurethe ground control points and determine that the ground control points308 a-308 d actually lie at the following coordinates relative to asection corner on the Earth: (1000, 3000, 10), (1001, 3011, 11), (1009,3009, 16) (1008, 999, 6). The georeferencing system 100 can generate andapply the transformation 320 that modifies (e.g., translates, rotates,and scales) the point cloud 310 such that the ground control points 308a-308 d are located at the measured survey coordinates relative to thesection corner on the Earth.

Similarly, the georeferencing system 100 can apply the transformation320 to the data points corresponding to the first structure 302, thesecond structure 304, and the road 306 so that the data points arelocated at their corresponding coordinates relative to the sectioncorner on the Earth. Thus, the georeferencing system 100 may build thepoint cloud 310 where the corner 304 a of the second structure 304 islocated at coordinate (7, 3, 0), and after applying the transformation320, the corner 304 a of the second structure 304 may be located atcoordinate (1007, 3003, 10) relative to the section corner on the Earth.

As discussed above, the georeferencing system 100 can also generategeoreference information for a set of images that do not show knownground control points. FIG. 3D illustrates a representation of a secondplurality of images 330 captured in accordance with one or moreembodiments disclosed herein. The second plurality of images 330 depictsthe same site as the plurality of images 300. However, in thisparticular embodiment, the georeferencing system 100 captures the secondplurality of images 330 days after the plurality of images 300, andafter changes to ground conditions have occurred. In particular, thesecond plurality of images 330 reflect a new structure 332, a structuraladdition 334 (i.e., an addition to the second structure 304); and anadditional road 336 (i.e., an addition to road 306).

In other embodiments, the georeferencing system 100 can capture thesecond plurality of images 330 at a variety of points in time. Forinstance, the georeferencing system 100 can capture the second pluralityof images 330 before the plurality of images 300. In particular, thegeoreferencing system 100 can capture the second plurality of images 330within hours, days, minutes, weeks, months, or years of the plurality ofimages 300.

Similarly, georeferencing system 100 may capture individual imageswithin the second plurality of images 330 at a variety of times. Withregard to FIG. 3D, the georeferencing system 100 captures the secondplurality of images 330 during the same flight of a UAV. In otherembodiments, however, the georeferencing system 100 may captureindividual images of the second plurality of images 330 on differentdays. For example, to provide greater detail with regard to a particularfeature, the georeferencing system 100 may utilize images showing theparticular feature (unchanged) captured on different days.

Notably, the second plurality of images 330 does not show the groundcontrol points 308 a-308 d. As described above, the georeferencingsystem 100 can generate georeference information related to a set ofimages that does not include known ground control points based on aprevious set of images that does include known ground control points.Specifically, and as discussed in more detail below, in one or moreembodiments the georeferencing system 100 generates a new point cloudutilizing both the plurality of images 300 and the second plurality ofimages 330. In this manner, the georeferencing system 302 preserves muchof the structure of point cloud 310, but incorporates new features ofthe second plurality of images 330. The georeferencing system 100applies the transformation 320 (e.g., the transformation previouslycalculated with regard to point cloud 310) to the new point cloud. Byapplying the transformation 320 to the new point cloud, thegeoreferencing system 100 creates a new georeferenced point cloud thatincorporates features from the second plurality of images (even thoughthe second plurality of images did not show ground control points).

Indeed, to that end FIG. 3E illustrates a representation of a pointcloud 340 generated based on the plurality of images 300 and the secondplurality of images 330. In particular, FIG. 3E illustratesrepresentations of data points corresponding to the first structure 302,the second structure 304, the new structure 332, the structural addition334, and the additional road 336.

With regard to FIG. 3E, the georeferencing system 100 generates thepoint cloud 340 by comparing the plurality of images 300 and the secondplurality of images 330. In particular, the georeferencing system 100identifies features in the plurality of images 300 and the secondplurality of images 330 and determines features that match. As describedabove, in one or more embodiments the georeferencing system 100generates a three-dimensional point cloud based on the matched features.For example, the georeferencing system 100 can calculate featuretrajectories and utilize the feature trajectories to constructthree-dimensional position of matching features. In particular, one ormore embodiments of the georeferencing system 100 utilize a structurefrom motion algorithm to generate the three-dimensional point cloud.

In one or more embodiments, the georeferencing system 100 utilizes anincremental bundle adjustment algorithm (as discussed above), adding thesecond plurality of images 330 to the plurality of images 300 in thebundle adjustment. For instance, by adding the second plurality ofimages 330 to the plurality of images 300, the georeferencing system 100can grow the reconstruction of the two-dimensional images to thethree-dimensional point cloud. Thus, the second plurality of images 330projects onto the three-dimensional structure (i.e., point cloud)established by the plurality of images 300. That said, because thegeoreferencing system 100 captures the plurality of images 300 and thesecond plurality of images 330 at different times (e.g., on differentdays), the georeferencing system 100 utilizes a filter to ensure properfeature matching.

To produce a point cloud reflecting changes in ground condition overtime, the georeferencing system 100 can prioritize one or more features.In particular, with regard to FIG. 3E, the georeferencing system 100prioritizes the features of the second plurality of images 330 toreflect the new features in the second plurality of images 330 in thepoint cloud 340. For example, the new structure 332 did not appear inthe plurality of images 300. The georeferencing system 100 prioritizesthe features of the second plurality of images 330 so that data pointsrelating to the new structure 332 appear in the point cloud 340.Similarly, the georeferencing system 100 prioritizes the features of thesecond plurality of images 330 so that data points relating to thestructural addition 334 and the additional road 336 (i.e., modificationsto an existing structure or feature) are reflected in the point cloud340.

In one or more embodiments, the georeferencing system 100 can prioritizeparticular features from one or more particular images (or sets ofimages). For example, in one or more embodiments, the georeferencingsystem 100 can prioritize ground control points so that the groundcontrol points 308 a-308 d appear in the point cloud 340. It is notnecessary, however, to operation of the georeferencing system 100 toensure that data points corresponding to the ground control points 308a-308 d appear in the point cloud 340. Thus, in the embodimentillustrated in FIG. 3E, the georeferencing system 100 generates thepoint cloud 340 without data points corresponding to known groundcontrol points.

In producing the point cloud 340 from the plurality of images 300 andthe second plurality of images 330, the georeferencing system 100 maypreserve coordinates relative to the point cloud 310. For instance, asdescribed above, the corner 304 a of the second structure 304 is locatedat coordinate (7, 3, 0) with regard to the point cloud 310 (prior toapplying the transformation 320). With regard to FIG. E, aftergenerating the point cloud 340 from the plurality of images 300 and thesecond plurality of images 330, the unchanged corner 304 a of the secondstructure 304 is still located at coordinate (7, 3, 0) in the pointcloud 340. In other words, the georeferencing system 100 can utilize thebundle adjustment to align the second plurality of images 330 to thestructures of the point cloud 310.

As discussed previously, the georeferencing system 100 may apply apreviously calculated transformation to a new point cloud. FIG. 3Fillustrates application of the transformation 320 to the point cloud340. The georeferencing system 100 applies the transformation 320 tomodify (i.e., rotate, translate, and scale), the point cloud 340. Inparticular, the georeferencing system 100 applies the transformation 320so that the data points in the point cloud 340 relate to correspondinggeographic coordinates (e.g., to corresponding locations on Earth).

For example, as discussed previously with regard to FIG. 3C, afterapplication of the transformation 320 to the point cloud 310, a datapoint corresponding to the corner 304 a of the second structure 304 waslocated at point (1007, 3003, 10) relative to a section corner on theEarth. Upon application of the transformation 320 to the point cloud340, a data point corresponding to the corner 304 a of the secondstructure 304 is also located at point (1007, 3003, 10) relative to thesection corner on the Earth. However, data points corresponding to newor modified features are now located with regard to new or modified realworld coordinates. For example, if corner 304 a of the second structure304 shifted to a modified location due to settlement as reflected in thesecond plurality of images 330 and the point cloud 340, the applicationof the transformation 320 to the point cloud 340 will produce a datapoint located at the modified location—e.g., (1007, 3003, 9). Thegeoreferencing system 100 can produce this georeference information,even though the second plurality of images 330 showed no ground controlpoints.

Although FIGS. 3A-3F illustrate the plurality of images 300 showingground control points, and generating a georeferenced point cloudutilizing ground control points, as mentioned previously, in one or moreembodiments, the georeferencing system 100 may capture a set of imageswith GPS data associated with the set of images. Moreover, in one ormore embodiments, the georeferencing system 100 can utilize GPS data togenerate a georeferenced point cloud. For example, in one or moreembodiments, georeferencing system 100 may capture the plurality ofimages 300 with GPS data providing the location of a UAV at the time oneor more of the plurality of images 300 were captured. The georeferencingsystem 100 can utilize GPS data to build a georeferenced point cloud.

In additional or alternative embodiments, however, the georeferencingsystem 100 may capture an image with GPS data, but not utilize the GPSdata for generating georeference information. For example, in somecircumstances (depending on features of a GPS system and other factors)ground control points may provide a more accurate reference and providemore accurate georeference information for a set of aerial images. Thus,in some embodiments the georeferencing system 100 may capture theplurality of images 300 showing the ground control points 308 a-308 dand also capture the plurality of images 300 with corresponding GPSdata. In such circumstances, the georeferencing system 100 can utilizeonly the ground control points 308-308 d, only the GPS data, or both togenerate a georeferenced point cloud.

Similarly, in some circumstances the georeferencing system 100 maycapture the second plurality of images 330 with GPS data, but withoutground control points. In one or more embodiments, the georeferencingsystem 100 will generate georeference information for the secondplurality of images 330 without utilizing the GPS data captured with thesecond plurality of images 330. Indeed, as mentioned, in someapplications, such an approach can provide even more accurategeoreference information than utilizing GPS data related to the secondplurality of images 330 to generate georeference information.Regardless, ultimately the georeferencing system can generategeoreference information with regard to the second plurality of images330 utilizing georeference information related to the plurality ofimages 300.

As mentioned previously, in some circumstances the process of generatinga point cloud from a plurality of images can result in drift.Specifically, iterative or sequential structure from motion processescan lead to accumulated inaccuracies in the point cloud. Accordingly, inone or more embodiments, the georeferencing system 100 controls for suchinaccuracies. In particular, one or more embodiments of thegeoreferencing system 100 refines ground control points and utilizes anadditional constrained bundle adjustment algorithm based on the refinedground control points to reduce drift.

More specifically, in one or more embodiments, the georeferencing system100 can generate refined ground control points by comparing observedpositions of ground control points to measured positions of the groundcontrol points. In particular, the georeferencing system 100 cangenerate refined ground control points by filtering observed positions,generating a transformation between the observed positions and measuredpositions of the ground control points, and applying the transformationinversely to generate refined ground control points that reflect, withina local coordinate system, the relative orientation, alignment, andposition of the measured positions of the ground control points.

For example, FIGS. 4A-4C illustrate generating refined ground controlpoints in accordance with one or more embodiments. In particular, FIG.4A illustrates a point cloud 400 showing observed positions 402 ofground control points with regard to a point cloud reference system 404(e.g., a relative coordinate system defining the location of theobserved positions 402 within the point cloud). With regard to theembodiment of FIG. 4A, the observed positions 402 are points in a pointcloud generated from a plurality of images showing ground controlpoints. Because of drift, the observed positions 402 do not matchmeasured positions of the ground control points. For instance, therelative location, alignment, and orientation of the observed positions402 may differ from the measured positions of the ground control points.

As mentioned above, in one or more embodiments, the georeferencingsystem 100 filters the observed positions 402. For instance, in one ormore embodiments the georeferencing system 100 applies a filter toidentify outliers (and correspondingly, inliers) from the observedpositions 402. Specifically, one or more embodiments of thegeoreferencing system 100 calculates error values associated with one ormore of the observed positions 402 and compares the error values to apre-determined threshold (e.g., identify outliers where the observedpositions 402 correspond to error values greater than the pre-determinedthreshold).

For example, one or more embodiments of the georeferencing system 100calculate an error value corresponding to one or more of the observedpositions 402 by generating a transformation for one or more of theobserved positions 402 with regard to measured positions of the groundcontrol points. The georeferencing system 100 can compare the accuracyof the transformation applied to the observed positions 402 relative tothe measured positions of the ground control points. For instance, thegeoreferencing system 100 can identify error values based on thedifference between the measured position of the ground control pointsand the observed positions 402 after application of a transformation,according to the following:E=T×GCP _(Observed) −GCP _(Measured)where E is the error value, T is the best fit transformation of theobserved positions relative to the measured positions, GCP_(Observed) isthe observed position of ground control points, and GCP_(Measured) isthe measured position of the ground control points.

Moreover the georeferencing system 100 can identify inliers and/oroutliers by comparing the error value to a threshold. For example, thegeoreferencing system 100 can identify outliers if the error value isgreater than a pre-determined error threshold and identify inliers ifthe error value is less than a predetermined threshold. Morespecifically, one or more embodiments of the georeferencing system 100utilize a RANSAC algorithm to identify outliers (and correspondinglyinliers) in light of error values corresponding to one or more observedpositions.

In one or more embodiments, the georeferencing system 100 can select athreshold (e.g., the pre-determined thresholds discussed above). Forinstance, in one or more embodiments, the georeferencing system 100identifies a threshold based on user input. Similarly, in one or moreembodiments, the georeferencing system 100 selects thresholds based on acharacteristic of a plurality of images or a three-dimensionalrepresentation (e.g., a resolution or density of a three-dimensionalpoint cloud). In other embodiments, the georeferencing system 100 canutilize other heuristics to select a threshold.

One or more embodiments of the georeferencing system 100 can also applyalternative or additional filters. For instance, in one or moreembodiments the georeferencing system 100 filters the observed positions402 that are based on an insufficient number of observations (e.g., aninsufficient number of aerial images showing the ground control point).For instance, if the ground control point does not have enoughobservations or some of its observations are false positives(observations that do not correspond to the ground control point), oneor more embodiments of the georeferencing system 100 filter the point ingenerating refined ground control points.

For example, FIG. 4B illustrates the point cloud 400 after applicationof a filter. In particular, FIG. 4B illustrates observed positions 402a-402 n that satisfy application of the filter described above.Specifically, the observed positions 402 a-402 n reflect inliers fromthe observed positions 402 that individually stem from a sufficientnumber of aerial images.

To generate refined ground control points, one or more embodiments ofthe georeferencing system 100 calculate a transformation that best fitsthe observed positions 402 a-402 n. Specifically, with regard to FIG.4B, the georeferencing system 100 identifies a transformation 410 thatbest fits the observed positions 402 a-402 n to measured positions 406on real-world reference system 408. The measured positions 406 representthe measured positions of the ground control points corresponding to theobserved positions 402 of the ground control points. Moreover, thereal-world reference system 408 reflects a relative coordinate systemutilized to describe the measured positions 406 of the ground controlpoints. Thus, the transformation 410 converts the observed positions 402a-402 n from the point cloud reference system 404 to the real-worldreference system 408.

As illustrated in FIG. 4B the transformation 410 may not perfectly fitthe observed positions 402 a-402 n to the corresponding measuredpositions 406. Indeed, drift may make a perfect transformation betweenthe observed positions 402 a-402 n to the measured positions 406unfeasible. Accordingly, in one or more embodiments, the transformation410 represents a best fit transformation between the observed positions402 a-402 n and the measured positions 406. Regardless, thetransformation 410 assists in converting the observed positions to thereal-world reference system 408.

The transformation 410 is not only useful in converting from the pointcloud reference system 404 to the real-world reference system 408, thetransformation can also be utilized to convert from the real-worldreference system 408 to the point cloud reference system 404. Indeed,the georeferencing system 100 can utilize the transformation 410inversely to convert the measured positions 406 from the real-worldreference system 408 to the point cloud reference system 404. Convertingpoints (e.g., measured positions of ground control points) from thereal-world reference system 408 to the point cloud reference system 404can assist in removing drift.

For example, as shown in FIG. 4C, the georeferencing system 100 utilizesthe transformation 410 inversely. In particular, the georeferencingsystem 100 utilizes the transformation 410 inversely to translate themeasured positions 406 to the point cloud reference system 404. In thismanner, the georeferencing system 100 can generate refined groundcontrol points 412. In particular, the refined ground control points 412represent the measured positions 406 in the point cloud reference system404. More specifically, the refined ground control points 412 preservethe relative orientation, alignment, and position of the measuredpositions 406 while converting the measured positions 406 to the pointcloud reference system 404.

As illustrated in FIG. 4C, in one or more embodiments, thegeoreferencing system 100 can convert the measured positions 406 to thepoint cloud reference system 404 (e.g., utilizing the transformation410) even if the measured positions 406 do not correspond to thefiltered, observed positions 402 a-402 n (i.e., the inliers). Forexample, measured point 406 a does not correspond to any of the observedpositions 402 a-402 n. One or more embodiments of the georeferencingsystem 100 can apply the transformation 410 inversely to the measuredpoint 406 a to convert the measured point 406 a to the point cloudreference system 404. In other words, the georeferencing system 100 canutilize the transformation 410 to convert any number of the measuredpoints 406 (or other real-world points) to the point cloud referencesystem 404. In other embodiments, the georeferencing system 100 canutilize the transformation 410 to only convert the measured positions406 that correspond to the observed positions 402 a-402 n (i.e., theinliers).

It will be appreciated that although the point cloud reference system404 and the real-world reference system 408 are illustrated with regardto FIGS. 4A-4C as being similar in rotation, scale, and location, thepoint cloud reference system 404 and real-world reference system 408 canbe differ in rotation (in any direction), scale (larger or smaller), andlocation (in any direction). Indeed, as discussed above, with regard tothe transformation 320, the transformation 410 can rotate, scale, andtranslate points (including observed positions 402 and measuredpositions 406) between reference systems that are arranged at anyorientation, scale, and translation.

Aside from calculating the refined ground control points 412, one ormore embodiments of the georeferencing system 100 utilizes the refinedground control points 412 to address drift in a point cloud. Inparticular, in one or more embodiments the georeferencing system 100utilizes the refined ground control points 412 as constraints (e.g.,anchor points) in remedying errors in a point cloud. Specifically, thegeoreferencing system 100 can utilize the refined ground control points412 as constraints in applying a constrained bundle adjustmentalgorithm.

As mentioned above, a bundle adjustment algorithm utilizes a pluralityof images to refine three-dimensional points describing a site andparameters regarding camera intrinsics (e.g., optical characteristics)and camera extrinsics (e.g., camera position). In other words, a bundleadjustment algorithm is an optimization problem with regard to thethree-dimensional structure of a site and parameters of a cameracapturing a plurality of images of the site. Moreover, a constrainedbundle adjustment algorithm is a bundle adjustment algorithm withapplied limitations (e.g., limitations or constraints based on knowninformation). For example, a constrained bundle adjustment algorithm canutilize given information (e.g., known camera parameters, knownthree-dimensional points, etc.) in optimizing remaining unknownthree-dimensional structure points and/or camera parameters.

In one or more embodiments, the georeferencing system 100 utilizes aconstrained bundle adjustment based on the refined ground control points412. Specifically, in one or more embodiments, the georeferencing system100 sets the refined ground control points 412 as constant constraints(e.g., defines the observed position of ground control points 406 basedon the refined ground control points 412). With the refined groundcontrol points 412 set as constant values, the georeferencing system 100can apply a constrained bundle adjustment to the remainder of a pointcloud corresponding to the ground control points. In this manner, thegeoreferencing system 100 can utilize the refined ground control points412 to rectify accumulated errors in the point cloud (e.g., utilize therefined ground control points as anchors in a constrained bundleadjustment algorithm to correct drift).

In addition to setting the refined ground control points 412 asconstant, the georeferencing system 100 can also utilize the refinedground control points 412 in a constrained bundle adjustment algorithmby weighting the ground control points. In particular, one or moreembodiments of the georeferencing system 100 weigh the ground controlpoints so that the bundle adjustment algorithm will emphasize errorsassociated with the refined ground control points 412 more heavily inoptimizing the point cloud structure and camera parameters.Specifically, one or more embodiments of the georeferencing system 100apply a scaling factor to the reprojection cost of the refined groundcontrol points 412 in applying a constrained bundle adjustmentalgorithm.

For example, one or more embodiments of the georeferencing system 100apply a constrained bundle adjustment that performs its optimizationbased on one or more cost functions. For instance, the bundle adjustmentcan utilize the cost function as a measure of the amount of error tooptimize (e.g., minimize). Accordingly, one or more embodiments of thegeoreferencing system 100 utilize the following cost function withregard to a constrained bundle adjustment algorithm to weight therefined ground control points 412:C=(s)[((K)[R|t](X))−x]where C is the cost, s is the scaling factor, K is the cameraintrinsics, [R|t] is the camera extrinsics, X is one or morethree-dimensional points (e.g., one or more points from the point cloud,such as ground control points), and x is the two-dimensional observationof one or more three-dimensional points (e.g., an image of one or morepoints from a plurality of images).

Accordingly, one or more embodiments set X as constant (e.g., set theground control points as constant based on the refined ground controlpoints 412), apply a heightened scaling factor (e.g., a heightenedscaling factor corresponding to the ground control points), and solvefor the optimal solution of the remaining variables.

The georeferencing system 100 can come to the optimal solution utilizinga variety of tools or models. One or more embodiments of thegeoreferencing system 100 utilizes a non-linear least square solver(e.g., Ceres Solver). For example, the georeferencing system can add thepoint cloud and ground control points into the non-linear least squaresolver with the cost function C defined above and solve for the optimalsolution. In this manner, the georeferencing system 100 can obtain acorrected point cloud.

Upon creation of an accurate georeferenced point cloud, thegeoreferencing system 100 can provide additional information to a user.For example, the georeferencing system 100 can provide, via a userinterface, a representation or model of a georeferenced point cloud.Moreover, in one or more embodiments the georeferencing system 100 cangenerate an orthomap from a georeferenced point cloud.

Furthermore, the georeferencing system 100 can utilize an orthomap tofurther generate georeference information. In particular, FIG. 5illustrates a representation of creating georeference informationutilizing an existing georeferenced orthomap (e.g., a georeferencedorthomap created from existing images, as just described). For example,FIG. 5 illustrates a georeferenced orthomap 500, portraying a firststructure 502, a second structure 504, and ground control points 508a-508 d.

Moreover, FIG. 5 illustrates a new image 510. In one or more embodimentsthe new image 510 is an image that can be approximated to a plane andutilized with a homography. The new image 510 shows the same locationportrayed in the georeferenced orthomap 500, albeit at a later point intime and after modifications to ground conditions. In particular, thenew image 510 portrays a structural modification 514 and a new road 516not shown in the georeferenced orthomap 500. Moreover, the new image 510does not show ground control points. Accordingly, the new image 510 isan unreferenced aerial image.

The georeferencing system 100 can generate a new georeferenced orthomapcontaining features of the new image 510 from the georeferenced orthomap500 and the new image 510. In particular, the georeferencing system 100can detect and compare features from the georeferenced orthomap 500 andthe new image 510. Moreover, the georeferencing system 100 can identifyone or more matching features from the georeferenced orthomap 500 andthe new image 510. Thus, with regard to FIG. 5, the georeferencingsystem 100 can identify features 509 a-509 d from the georeferencedorthomap 500 that respectively match features 512 a-512 d from new image510.

As discussed earlier, the georeferencing system 100 can generate ahomography, such as homography 520, illustrated in FIG. 5. Inparticular, with regard to FIG. 5, the georeferencing system generatesthe homography 520 based on, at least, the matched features 509 a-509 dand 512 a-512 d. Based on the matched features, the homography 520enables the georeferencing system 100 to map features from the new image510 to the georeferenced orthomap 500.

As discussed above, the georeferencing system 100 can prioritize one ormore features. Accordingly, the georeferencing system 100 can prioritizethe features of new image 510. For example, FIG. 5 illustrates a newgeoreferenced orthomap 530 generated using the homography 520 (based onthe georeferenced orthomap 500 and the new image 510). As illustrated,the new georeferenced orthomap 530 displays the structural modification514 shown in the new image 510 and the new road 516 shown in the newimage 510. The new georeferenced orthomap 530 permits a user to measureprecise distances and locations from the ground control points 508 a-508d with regard to features and modifications shown in the new image 510,despite the fact that the new image 510 does not show ground controlpoints.

Moreover, as shown, the georeferencing system 100 converts features fromthe new image 510 to a new georeferenced orthomap 530. After mappingfeatures of the new image 510 utilizing the homography 520 and thegeoreferenced orthomap 500, the new georeferenced orthomap 530 includesthe features of the new image 510.

The georeferencing system 100 can also generate georeference informationwithout a georeferenced orthomap. FIG. 6 illustrates a representation ofgenerating georeference information from a georeferenced aerial image inaccordance with one or more embodiments. In particular, FIG. 6illustrates a georeferenced aerial image 600, showing a first structure602, a second structure 604, a road 606, and ground control points 608a-608 d. In one or more embodiments georeferenced aerial image 600comprises a plurality of aerial images. Not all of the images in thegeoreferenced aerial image 600 need show all the ground control points608 a-608 d, the first structure 602, the second structure 604, and theroad 606; to the contrary, each image may only show a portion of a site.For example, each image of the georeferenced aerial image 600 may show asingle ground control point. In one or more embodiments, thegeoreferenced aerial image 600 comprises one or more images that can beapproximated to a plane for utilization with a homography, as discussedbelow.

FIG. 6 also illustrates a new image 610. As shown, the new image 610does not show ground control points, but does show a new road 612 and anew structure 614. In one or more embodiments, the new image 610 cancomprise a plurality of aerial images. Each image in the plurality ofaerial images from the new image 610 may show only a portion of a site;for example, one image may only show the first structure 602 and anotherimage may only show the new road 612 (or some other portion). Moreover,in one or more embodiments, the georeferenced aerial image 600 comprisesone or more images that can be approximated to a plan for utilization inconjunction with a homography.

The georeferencing system 100 detects features from a georeferencedaerial image 600 and the new image 610. The georeferencing system 100compares detected features and identifies features common to thegeoreferenced aerial image 600 and the new image 610. In one or moreembodiments, the georeferencing system 100 can identify one or morecommon features or one or more new features based on user input. Forexample, the georeferencing system 100 can receive user inputidentifying the first structure 602 as a common feature between thegeoreferenced aerial image 600 and new image 610. Additionally oralternatively, the georeferencing system 100 can autonomously identifycommon features.

As discussed previously, the georeferencing system 100 can utilize ahomography 620 to generate georeference information. In particular,based on common features of the georeferenced aerial image 600 and thenew image 610 the georeferencing system 100 can utilize the homography620 to generate georeference information for a new georeferenced aerialimage 630. As discussed previously, in one or more embodiments, thegeoreferencing system 100 estimates a homography robustly between thetwo images to account for variations. The georeferencing system 100 canutilize the homography 620 to generate georeference information withregard to the new image 610.

For example, with regard to FIG. 6, the georeferencing system 100detects coordinates associated with the ground control points 608 a-608d. The georeferencing system 100 utilizes the homography 620 to computethe position of the ground control points 608 a-608 d with regard to newimage 610. Thus, the georeferencing system 100 can generate the newgeoreferenced aerial image 630 containing the ground control points 608a-608 d.

In one or more embodiments, the georeferencing system 100 projects onlythe ground points 608 a-608 d onto the new image 610 to produce the newgeoreferenced aerial image 630. In other embodiments, the georeferencingsystem 100 projects other points onto the new image 610 to produce thenew georeferenced aerial image 630. In other embodiments, thegeoreferencing system 100 projects points from the new image 610 to thegeoreferenced aerial image 600 to produce the new georeferenced aerialimage 630. Regardless, at bottom the georeferencing system 100 canutilize a homography to produce georeferenced aerial image containingfeatures of an aerial image captured without georeference information.

Although the exemplary embodiments illustrated and discussed with regardto FIGS. 3A-6 have illustrated a certain number of ground controlpoints, it will be appreciated that the number of ground control pointsmay vary. For example, in one or more embodiments, images may containgreater or fewer ground control points than illustrated. Similarly, withregard to embodiments utilizing a plurality of images to generategeoreference information, some of the individual images in the pluralityof images may not show ground control points.

In some circumstances a minimum number of ground control points may beneeded to perform the functions of the georeferencing system 100. Forexample, as discussed, the georeferencing system 100 can utilize a firstaerial image with ground control points to generate georeferenceinformation for a second aerial image captured without ground controlpoints. In such circumstances, one or more embodiments of thegeoreferencing system 100 utilize at least three ground control pointsto generate the new georeferenced aerial image.

As mentioned previously, in one or more embodiments, the georeferencingsystem 100 utilizes georeference information for navigation. FIG. 7illustrates a representation of utilizing georeference information fornavigation in accordance with one or more embodiments. FIG. 7illustrates a UAV 702 in flight over a site containing a structure 704,a road 706, a structural modification 708, and a new road 710. The UAV702 includes an onboard digital camera capable of capturing digitalaerial images during flight.

In the embodiment illustrated in FIG. 7, the georeferencing system 100accesses an existing georeferenced orthomap for utilization duringflight. Moreover, the UAV 702 captures a new aerial image of the siteduring flight. The georeferencing system 100 then utilizes the existinggeoreferenced orthomap and the new aerial image to generate a newgeoreferenced orthomap reflecting features of the new aerial image (asdescribed above). The UAV 702 can then navigate in flight utilizing thenew georeferenced orthomap.

For example, based on the new georeferenced orthomap, the UAV 702 candetermine the location of the UAV 702 in relation to the Earth.Similarly, the UAV 702 can determine the location of one or more objectsutilizing the new georeferenced orthomap. For example, the UAV 702 candetermine the location of the structure 704 utilizing the newgeoreferenced orthomap, and navigate to the location of the structure704 based on the new georeferenced orthomap.

In one or more embodiments, all or part of the georeferencing system 100can be implemented as part of a computer system aboard the UAV 702.Thus, in one or more embodiments, the georeferencing system 100 can loadan existing georeferenced orthomap on a computer system aboard the UAV702. Moreover, in one or more embodiments of the georeferencing system100, a camera operatively coupled to the UAV 702 captures a new image, acomputer system aboard the UAV 702 generates a new georeferencedorthomap, and the computer system aboard the UAV 702 navigates the UAV702 based on the new georeferenced orthomap. In other embodiments,portions of the georeferencing system 100 are implemented as part of theUAV 702 and other portions are implemented via other components (e.g., aground station or a server) as described above with regard to FIG. 2.

The georeferencing system 100 can navigate based on detected locationinformation, features, objects, similarities, differences, changes,modifications, or other information. For instance, in one or moreembodiments, the georeferencing system 100 navigates based ondifferences between an existing georeferenced orthomap and one or morenew images (or the difference between an existing georeferenced orthomapand a new georeferenced orthomap). As illustrated in FIG. 7, the UAV 702detects the structural modification 708, and navigates based on thedetected modification (e.g., navigates toward the structuralmodification 708 to take additional aerial images). Similarly, the UAV702 detects the new road 710 not previously present in an existinggeoreferenced orthomap and navigates based on the detected modification.

In other embodiments, the georeferencing system 100 can provide a newgeoreferenced orthomap (generated based on a new image captured inflight) to one or more individuals navigating the UAV 702. For example,the georeferencing system 100 can provide for display, via a displaydevice, a new georeferenced orthomap to an individual to allow theindividual to more accurately navigate the UAV 702. For instance, thegeoreferencing system 100 can provide a user interface displaying ageoreferenced orthomap (generated based on a new image captured inflight) with an indication of the location of the UAV 702, so that anindividual navigating the UAV 702 can determine the precise location ofthe UAV 702 and navigate the UAV 702 accordingly.

For example, an individual navigating the UAV 702 from the ground maybelieve that the UAV 702 is flying (and capturing images) directly abovethe structure 704. A new georeferenced orthomap generated by thegeoreferencing system 100 may reveal that the UAV 702 is not directlyover the structure 704 and allow the individual navigating the UAV 702to modify the location of the UAV 702 accordingly.

The georeferencing system 100 can also utilize generated georeferenceinformation with regard to design. FIG. 8A illustrates a computer system800 including a display device 802 with a display area 804. Asillustrated, the display area 804 shows a georeferenced orthomap 706(represented by solid lines) and proposed design elements (representedby dashed lines). In particular, the proposed design elements include anew structure 808, a utility addition 810, and a new waterline 812.Thus, the georeferencing system 100 can generate a georeferencedorthomap that provides to-scale measurements for utilization inengineering design.

As discussed, however, the georeferencing system 100 can also generategeoreference information over time based on new images captured withoutgeoreference information and provide updated information for display toa user. FIG. 8B illustrates the computer system 800 displaying (via thedisplay device 802) a new georeferenced orthomap 820 in conjunction withthe design elements 708-712. In the embodiment illustrated in FIG. 8B,the georeferencing system 100 generates the new georeferenced orthomap820 (i.e., generates georeference information) based on a plurality ofnew images captured without georeference information; then, utilizingthe generated georeference information, provides the new georeferencedorthomap 820 in relation to the design elements 808-812.

By providing the new georeferenced orthomap 820 in conjunction with thedesign elements 808-812, the georeferencing system 100 permits a user toview the design in relation to up-to-date and accurate fieldinformation. For example, as illustrated in FIG. 8B, a user can see thata construction crew has opened a trench 822 for the utility addition810, but has not completed the trench 822. Similarly, a user can seethat footings 824 a-824 d have been constructed in the proper location,but that the remainder of the new structure 808 remains uncompleted. Inaddition, a user can see that a construction crew has opened a trench826 that is not in alignment with the designed location for the newwaterline 812. Accordingly, the user can make appropriate decisionsbased on the comparison (e.g., call the construction crew to determinethe reason for the change in alignment).

The user can also make design changes, additions, or revisions based onthe georeference information generated by the georeferencing system 100.For example, based on the location of the trench 826, an engineer candesign a revised waterline.

Although FIGS. 8A, 8B illustrate providing georeferenced orthomaps via adisplay device, it will be appreciated that the georeferencing system100 can provide a variety of images, information, reports, or summariesto a user in a variety of forms. For example, the georeferencing system100 can generate engineering plans, as-built plans, models,three-dimensional representations, tables, or other information to auser. Thus, georeference information generated by the georeferencingsystem 100 can provide for display information that permits a user tomake design or policy decisions regarding issues related to navigatingthe site, modifying the site, building a structure at the site, or anyother issues.

FIGS. 1-8, the corresponding text, and the examples, provide a number ofdifferent systems and devices for generating georeference information.In addition to the foregoing, one or more embodiments can also bedescribed in terms of flowcharts comprising acts and steps in a methodfor accomplishing a particular result. For example, FIGS. 9 and 10illustrate flowcharts of exemplary methods in accordance with one ormore embodiments. The methods described in relation to FIGS. 9 and 10may be performed with less or more steps/acts or the steps/acts may beperformed in differing orders. Additionally, the steps/acts describedherein may be repeated or performed in parallel with one another or inparallel with different instances of the same or similar steps/acts.

FIG. 9 illustrates a flowchart of one example method 900 of generatinggeoreference information in accordance with one or more embodiments. Asillustrated, the method 900 may include the act 902 of accessing aerialimages showing ground control points. In particular, the act 902 mayinclude accessing a first plurality of aerial images showing a site, thefirst plurality of aerial images showing a plurality of known groundcontrol points. For example, in one or more embodiments the known groundcontrol points may comprise temporary markers placed at determinedlocations.

As shown in FIG. 9, the method 900 may also include an act 904 ofbuilding an unreferenced three-dimensional representation. Inparticular, the act 904 may include building an unreferencedthree-dimensional representation from a first plurality of aerialimages. For instance, the act 904 may include utilizing a structure frommotion algorithm and a bundle adjustment algorithm to convert the firstplurality of aerial images into an unreferenced three-dimensionalrepresentation representative of the site.

The act 904 may also include generating a plurality of refined groundcontrol points based on observed positions of the plurality of groundcontrol points in the plurality of aerial images and measured positionsof the plurality of ground control points, and applying a constrainedbundle adjustment algorithm to the initial three-dimensionalrepresentation utilizing the refined ground control points, wherein theconstrained bundle adjustment algorithm weights the refined groundcontrol points. Moreover, the act 904 may also include filtering theobserved positions of the plurality of ground control points based onthe measured positions of the plurality of ground control points;generating a transformation based on the filtered, observed positionsand the corresponding measured positions; and applying thetransformation inversely to the measured ground control points togenerate refined ground control points

Furthermore, as illustrated in FIG. 9, the method 900 may include an act906 of generating a transformation. In particular, the act 906 mayinclude generating, by at least one processor, a transformation based onthe unreferenced three-dimensional representation and the plurality ofknown ground control points. For example, the act 906 may includegenerating a transformation by utilizing a Helmert transformationalgorithm.

In addition, as shown in FIG. 9, the method 900 may include an act 908of accessing a new aerial image without ground control points. Inparticular, the act 908 may include accessing at least one new aerialimage showing the site, wherein the plurality of known ground controlpoints are not shown within the at least one new aerial image. The newaerial image may be captured at a different time than the aerial imagesshowing ground control points. For example, in one or more embodiments,the first plurality of aerial images are captured on a first day and theat least one new aerial image is captured on a day subsequent to thefirst day.

As illustrated in FIG. 9, the method 900 may also include an act 910 ofbuilding a new unreferenced three-dimensional representation based onthe new aerial image. In particular, the act 910 may include building,by at least one processor, a new unreferenced three-dimensionalrepresentation based, at least in part, on at least one new aerialimage. For example, the act 910 may include utilizing a structure frommotion algorithm and a bundle adjustment algorithm to convert the atleast one new image into a new unreferenced three-dimensionalrepresentation representative of the site. Similarly, the act 910 mayalso include identifying one or more common features between the firstplurality of aerial images and the at least one new aerial image andgenerating a new unreferenced three-dimensional representation based onthe common features. Furthermore, the act 910 may comprise prioritizingfeature from the at least one new aerial image such that a newunreferenced three-dimensional representation reflects at least onefeature from the at least one new aerial image not reflected in theunreferenced three-dimensional representation.

Moreover, as shown in FIG. 9, the method 900 may also include the act912 of applying the transformation to the new unreferencedthree-dimensional representation. In particular, the act 912 may includeapplying, by at least one processor, the transformation to the newunreferenced three-dimensional representation to create a newgeoreferenced three-dimensional representation. Furthermore, the act 912may also include generating a georeferenced orthomap from the newgeoreferenced three-dimensional representation. Moreover, the act 912may include, providing the georeferenced orthomap for utilization in atleast one of navigating the site, modifying the site, modifying at leasta portion of the site, or building a structure on at least a portion ofthe site.

FIG. 10 illustrates a flowchart of another example method 1000 ofgenerating georeference information in accordance with one or moreembodiments. As illustrated, the method 1000 includes an act 1002 ofaccessing a first aerial image showing ground control points. Inparticular, the act 1002 may include accessing at least one first aerialimage showing a site, the at least one first aerial image showing aplurality of known ground control points. Moreover, the act 1002 mayalso include, accessing a georeferenced orthomap showing a site, whereinthe georeferenced orthomap is based on at least one aerial image and aplurality of ground control points shown within the at least one aerialimage.

As illustrated in FIG. 10, the method 1000 may also include an act 1004of accessing a second aerial image without ground control points. Inparticular, the act 1004 may include accessing at least one secondaerial image of the site, the at least one second aerial image notshowing known ground control points. Similarly, the act 1004 may includeaccessing at least one new aerial image showing the site, wherein theplurality of ground control points are not shown within the at least onenew aerial image. In one or more embodiments, the act 1004 includescapturing the at least one new aerial image using one or more camerasonboard a UAV during a flight.

Moreover, as shown in FIG. 10, the method 1000 may also include an act1006 of comparing features from the first aerial image and the secondaerial image. In particular, the act 1006 may include comparing, by atleast one processor, one or more features from the at least one firstaerial image with one or more features from the at least one secondaerial image. For example, the act 1006 may include identifying commonfeatures between the at least one first aerial image and the at leastone second aerial image based on the comparison between the one or morefeatures from the at least one aerial image and the one or more featuresfrom the at least one second aerial image. The act 1006 may also includeestimating a homography based on the identified common features.

Similarly, the act 1006 may include matching one or more features fromthe georeferenced orthomap with one or more features from the at leastone new aerial image. For instance, the act 1006 may comprise detectingone or more features from the georeferenced orthomap; detecting one ormore features from the at least one new aerial image, and comparing theone or more detected features from the georeferenced orthomap with theone or more detected features from the at least one new aerial image.Moreover, the act 1006 may include computing a homography relating thegeoreferenced orthomap and the at least one new aerial image, based onthe one or more matched features from the georeferenced orthomap and theat least one new aerial image.

As illustrated in FIG. 10, the method 1000 may also include the act 1008of generating georeference information for the second aerial image. Inparticular, the act 1008 may include generating, by the at least oneprocessor and based on the comparison between the one or more featuresfrom the first plurality of aerial images and the one or more featuresfrom the at least one second aerial image, georeference information forthe at least one second aerial image. For instance, the act 1006 mayinclude identifying the location of at least one of the plurality ofground control points with regard to the at least one second aerialimage. Similarly, the act 1008 may also include the act of updating,utilizing the homography, the georeferenced orthomap based on the atleast one new aerial image.

The method 1000 may also include navigating a UAV during a flight basedon an updated georeferenced orthomap. For instance, the act 1008 mayinclude comparing the georeferenced orthomap and the updatedgeoreferenced orthomap and navigating the UAV based on the comparison.Similarly, the act 1008 may include identifying a modification to thesite based on the updated georeferenced orthomap and navigating the UAVbased on the modification to the site.

Embodiments of the present disclosure may comprise or utilize a specialpurpose or general-purpose computer including computer hardware, suchas, for example, one or more processors and system memory, as discussedin greater detail below. Embodiments within the scope of the presentdisclosure also include physical and other computer-readable media forcarrying or storing computer-executable instructions and/or datastructures. In particular, one or more of the processes described hereinmay be implemented at least in part as instructions embodied in anon-transitory computer-readable medium and executable by one or morecomputing devices (e.g., any of the media content access devicesdescribed herein). In general, a processor (e.g., a microprocessor)receives instructions, from a non-transitory computer-readable medium,(e.g., a memory, etc.), and executes those instructions, therebyperforming one or more processes, including one or more of the processesdescribed herein.

Computer-readable media can be any available media that can be accessedby a general purpose or special purpose computer system.Computer-readable media that store computer-executable instructions arenon-transitory computer-readable storage media (devices).Computer-readable media that carry computer-executable instructions aretransmission media. Thus, by way of example, and not limitation,embodiments of the disclosure can comprise at least two distinctlydifferent kinds of computer-readable media: non-transitorycomputer-readable storage media (devices) and transmission media.

Non-transitory computer-readable storage media (devices) includes RAM,ROM, EEPROM, CD-ROM, solid state drives (“SSDs”) (e.g., based on RAM),Flash memory, phase-change memory (“PCM”), other types of memory, otheroptical disk storage, magnetic disk storage or other magnetic storagedevices, or any other medium which can be used to store desired programcode means in the form of computer-executable instructions or datastructures and which can be accessed by a general purpose or specialpurpose computer.

A “network” is defined as one or more data links that enable thetransport of electronic data between computer systems and/or modulesand/or other electronic devices. When information is transferred orprovided over a network or another communications connection (eitherhardwired, wireless, or a combination of hardwired or wireless) to acomputer, the computer properly views the connection as a transmissionmedium. Transmissions media can include a network and/or data linkswhich can be used to carry desired program code means in the form ofcomputer-executable instructions or data structures and which can beaccessed by a general purpose or special purpose computer. Combinationsof the above should also be included within the scope ofcomputer-readable media.

Further, upon reaching various computer system components, program codemeans in the form of computer-executable instructions or data structurescan be transferred automatically from transmission media tonon-transitory computer-readable storage media (devices) (or viceversa). For example, computer-executable instructions or data structuresreceived over a network or data link can be buffered in RAM within anetwork interface module (e.g., a “NIC”), and then eventuallytransferred to computer system RAM and/or to less volatile computerstorage media (devices) at a computer system. Thus, it should beunderstood that non-transitory computer-readable storage media (devices)can be included in computer system components that also (or evenprimarily) utilize transmission media.

Computer-executable instructions comprise, for example, instructions anddata which, when executed at a processor, cause a general purposecomputer, special purpose computer, or special purpose processing deviceto perform a certain function or group of functions. In someembodiments, computer-executable instructions are executed on ageneral-purpose computer to turn the general purpose computer into aspecial purpose computer implementing elements of the disclosure. Thecomputer executable instructions may be, for example, binaries,intermediate format instructions such as assembly language, or evensource code. Although the subject matter has been described in languagespecific to structural features and/or methodological acts, it is to beunderstood that the subject matter defined in the appended claims is notnecessarily limited to the described features or acts described above.Rather, the described features and acts are disclosed as example formsof implementing the claims.

Those skilled in the art will appreciate that the disclosure may bepracticed in network computing environments with many types of computersystem configurations, including, personal computers, desktop computers,laptop computers, message processors, hand-held devices, multi-processorsystems, microprocessor-based or programmable consumer electronics,network PCs, minicomputers, mainframe computers, mobile telephones,PDAs, tablets, pagers, routers, switches, and the like. The disclosuremay also be practiced in distributed system environments where local andremote computer systems, which are linked (either by hardwired datalinks, wireless data links, or by a combination of hardwired andwireless data links) through a network, both perform tasks. In adistributed system environment, program modules may be located in bothlocal and remote memory storage devices.

Embodiments of the present disclosure can also be implemented in cloudcomputing environments. In this description, “cloud computing” isdefined as a model for enabling on-demand network access to a sharedpool of configurable computing resources. For example, cloud computingcan be employed in the marketplace to offer ubiquitous and convenienton-demand access to the shared pool of configurable computing resources.The shared pool of configurable computing resources can be rapidlyprovisioned via virtualization and released with low management effortor service provider interaction, and then scaled accordingly.

A cloud-computing model can be composed of various characteristics suchas, for example, on-demand self-service, broad network access, resourcepooling, rapid elasticity, measured service, and so forth. Acloud-computing model can also expose various service models, such as,for example, Software as a Service (“SaaS”), Platform as a Service(“PaaS”), and Infrastructure as a Service (“IaaS”). A cloud-computingmodel can also be deployed using different deployment models such asprivate cloud, community cloud, public cloud, hybrid cloud, and soforth. In this description and in the claims, a “cloud-computingenvironment” is an environment in which cloud computing is employed.

FIG. 11 illustrates a block diagram of an exemplary computing device1100 that may be configured to perform one or more of the processesdescribed above. One will appreciate that the georeferencing system maybe implemented by one or more computing devices such as the computingdevice 1100. As shown by FIG. 11, the computing device 1100 can comprisea processor 1102, memory 1104, a storage device 1106, an I/O interface1108, and a communication interface 1110, which may be communicativelycoupled by way of a communication infrastructure 1112. While anexemplary computing device 1100 is shown in FIG. 11, the componentsillustrated in FIG. 11 are not intended to be limiting. Additional oralternative components may be used in other embodiments. Furthermore, incertain embodiments, the computing device 1100 can include fewercomponents than those shown in FIG. 11. Components of the computingdevice 1100 shown in FIG. 11 will now be described in additional detail.

In particular embodiments, the processor 1102 includes hardware forexecuting instructions, such as those making up a computer program. Asan example and not by way of limitation, to execute instructions, theprocessor 1102 may retrieve (or fetch) the instructions from an internalregister, an internal cache, the memory 1104, or the storage device 1106and decode and execute them. In particular embodiments, the processor1102 may include one or more internal caches for data, instructions, oraddresses. As an example and not by way of limitation, the processor1102 may include one or more instruction caches, one or more datacaches, and one or more translation lookaside buffers (TLBs).Instructions in the instruction caches may be copies of instructions inthe memory 1104 or the storage 1106.

The memory 1104 may be used for storing data, metadata, and programs forexecution by the processor(s). The memory 1104 may include one or moreof volatile and non-volatile memories, such as Random Access Memory(“RAM”), Read Only Memory (“ROM”), a solid state disk (“SSD”), Flash,Phase Change Memory (“PCM”), or other types of data storage. The memory1104 may be internal or distributed memory.

The storage device 1106 includes storage for storing data orinstructions. As an example and not by way of limitation, the storagedevice 1106 can comprise a non-transitory storage medium describedabove. The storage device 1106 may include a hard disk drive (HDD), afloppy disk drive, flash memory, an optical disc, a magneto-opticaldisc, magnetic tape, or a Universal Serial Bus (USB) drive or acombination of two or more of these. The storage device 1106 may includeremovable or non-removable (or fixed) media, where appropriate. Thestorage device 1106 may be internal or external to the computing device1100. In particular embodiments, the storage device 1106 isnon-volatile, solid-state memory. In other embodiments, the storagedevice 1106 includes read-only memory (ROM). Where appropriate, this ROMmay be mask programmed ROM, programmable ROM (PROM), erasable PROM(EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM(EAROM), or flash memory or a combination of two or more of these.

The I/O interface 1108 allows a user to provide input to, receive outputfrom, and otherwise transfer data to and receive data from the computingdevice 1100. The I/O interface 1108 may include a mouse, a keypad or akeyboard, a touch screen, a camera, an optical scanner, networkinterface, modem, other known I/O devices or a combination of such I/Ointerfaces. The I/O interface 1108 may include one or more devices forpresenting output to a user, including, but not limited to, a graphicsengine, a display (e.g., a display screen), one or more output drivers(e.g., display drivers), one or more audio speakers, and one or moreaudio drivers. In certain embodiments, the I/O interface 1108 isconfigured to provide graphical data to a display for presentation to auser. The graphical data may be representative of one or more graphicaluser interfaces and/or any other graphical content as may serve aparticular implementation.

The communication interface 1110 can include hardware, software, orboth. In any event, the communication interface 1110 can provide one ormore interfaces for communication (such as, for example, packet-basedcommunication) between the computing device 1100 and one or more othercomputing devices or networks. As an example and not by way oflimitation, the communication interface 1110 may include a networkinterface controller (NIC) or network adapter for communicating with anEthernet or other wire-based network or a wireless NIC (WNIC) orwireless adapter for communicating with a wireless network, such as aWI-FI.

Additionally or alternatively, the communication interface 1110 mayfacilitate communications with an ad hoc network, a personal areanetwork (PAN), a local area network (LAN), a wide area network (WAN), ametropolitan area network (MAN), or one or more portions of the Internetor a combination of two or more of these. One or more portions of one ormore of these networks may be wired or wireless. As an example, thecommunication interface 1110 may facilitate communications with awireless PAN (WPAN) (such as, for example, a BLUETOOTH WPAN), a WI-FInetwork, a WI-MAX network, a cellular telephone network (such as, forexample, a Global System for Mobile Communications (GSM) network), orother suitable wireless network or a combination thereof.

Additionally, the communication interface 1110 may facilitatecommunications various communication protocols. Examples ofcommunication protocols that may be used include, but are not limitedto, data transmission media, communications devices, TransmissionControl Protocol (“TCP”), Internet Protocol (“IP”), File TransferProtocol (“FTP”), Telnet, Hypertext Transfer Protocol (“HTTP”),Hypertext Transfer Protocol Secure (“HTTPS”), Session InitiationProtocol (“SIP”), Simple Object Access Protocol (“SOAP”), ExtensibleMark-up Language (“XML”) and variations thereof, Simple Mail TransferProtocol (“SMTP”), Real-Time Transport Protocol (“RTP”), User DatagramProtocol (“UDP”), Global System for Mobile Communications (“GSM”)technologies, Code Division Multiple Access (“CDMA”) technologies, TimeDivision Multiple Access (“TDMA”) technologies, Short Message Service(“SMS”), Multimedia Message Service (“MMS”), radio frequency (“RF”)signaling technologies, Long Term Evolution (“LTE”) technologies,wireless communication technologies, in-band and out-of-band signalingtechnologies, and other suitable communications networks andtechnologies.

The communication infrastructure 1112 may include hardware, software, orboth that couples components of the computing device 1100 to each other.As an example and not by way of limitation, the communicationinfrastructure 1112 may include an Accelerated Graphics Port (AGP) orother graphics bus, an Enhanced Industry Standard Architecture (EISA)bus, a front-side bus (FSB), a HYPERTRANSPORT (HT) interconnect, anIndustry Standard Architecture (ISA) bus, an INFINIBAND interconnect, alow-pin-count (LPC) bus, a memory bus, a Micro Channel Architecture(MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express(PCIe) bus, a serial advanced technology attachment (SATA) bus, a VideoElectronics Standards Association local (VLB) bus, or another suitablebus or a combination thereof.

In the foregoing specification, the present disclosure has beendescribed with reference to specific exemplary embodiments thereof.Various embodiments and aspects of the present disclosure(s) aredescribed with reference to details discussed herein, and theaccompanying drawings illustrate the various embodiments. Thedescription above and drawings are illustrative of the disclosure andare not to be construed as limiting the disclosure. Numerous specificdetails are described to provide a thorough understanding of variousembodiments of the present disclosure.

The present disclosure may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. For example, the methods described herein may beperformed with less or more steps/acts or the steps/acts may beperformed in differing orders. Additionally, the steps/acts describedherein may be repeated or performed in parallel with one another or inparallel with different instances of the same or similar steps/acts. Thescope of the present application is, therefore, indicated by theappended claims rather than by the foregoing description. All changesthat come within the meaning and range of equivalency of the claims areto be embraced within their scope.

We claim:
 1. A computer-implemented method comprising: accessing a firstplurality of aerial images of a site, the first plurality of aerialimages comprising georeference information for the site; building aninitial three-dimensional representation of the site from the firstplurality of aerial images; generating, using at least one processor, atransformation based on the initial three-dimensional representation ofthe site and the georeference information for the site; accessing aplurality of new aerial images of the site without georeferenceinformation; building, by the at least one processor, a new unreferencedthree-dimensional representation of the site based, at least in part, onthe plurality of new aerial images of the site without georeferenceinformation; and applying, by the at least one processor, thetransformation generated based on the initial three-dimensionalrepresentation of the site and the georeference information for the siteto the new unreferenced three-dimensional representation built based onthe plurality of new aerial images of the site without georeferenceinformation to create a new georeferenced three-dimensionalrepresentation of the site.
 2. The method of claim 1, further comprisinggenerating a georeferenced orthomap from the new georeferencedthree-dimensional representation, wherein the georeferenced orthomapcomprises a geometrically corrected aerial image of the site where tiltand relief have been corrected to portray a uniform scale.
 3. The methodof claim 1, wherein building the initial three-dimensionalrepresentation of the site from the first plurality of aerial imagescomprises utilizing a structure from motion algorithm and a bundleadjustment algorithm to convert the first plurality of aerial imagesinto the initial three-dimensional representation of the site, andwherein building the new unreferenced three-dimensional representationof the site comprises utilizing the structure from motion algorithm andthe bundle adjustment algorithm to convert the plurality of new aerialimages of the site into the new unreferenced three-dimensionalrepresentation of the site.
 4. The method of claim 1, wherein the firstplurality of aerial images are captured at a first point in time withthe georeference information for the site and the plurality of newaerial images are captured at a second point in time subsequent to thefirst point in time without georeference information.
 5. The method ofclaim 1, wherein building the new unreferenced three-dimensionalrepresentation further comprises: identifying one or more commonfeatures between the first plurality of aerial images and the pluralityof new aerial images; and generating the new unreferencedthree-dimensional representation of the site based on the one or morecommon features.
 6. The method of claim 1, wherein building the newunreferenced three-dimensional representation of the site furthercomprises prioritizing features from the plurality of new aerial imagessuch that the new unreferenced three-dimensional representation of thesite reflects at least one feature from the plurality of new aerialimages not reflected in the initial three-dimensional representation ofthe site.
 7. The method of claim 2, further comprising providing thegeoreferenced orthomap for utilization in at least one of navigating thesite, modifying at least a portion of the site, or building a structureon at least a portion of the site.
 8. The method of claim 1, wherein thegeoreference information for the site comprises known ground controlpoints portrayed in the first plurality of aerial images.
 9. The methodof claim 1, wherein the georeference information for the site comprisesinformation obtained from a global positioning system during a period oftime when a digital camera captured the first plurality of aerialimages.
 10. The method of claim 9, wherein the plurality of new aerialimages of the site lack location information from the global positioningsystem.
 11. A system comprising: at least one processor; and at leastone non-transitory computer readable storage medium storing instructionsthat, when executed by the at least one processor, cause the system to:building an initial three-dimensional representation of a site from afirst plurality of aerial images of the site, the first plurality ofaerial images comprising georeference information for the site; generatea transformation based on the initial three-dimensional representationof the site and the georeference information for the site; access aplurality of new aerial images of the site without georeferenceinformation; build a new unreferenced three-dimensional representationof the site based, at least in part, on the plurality of new aerialimages of the site without georeference information; and apply thetransformation generated based on the initial three-dimensionalrepresentation of the site and the georeference information for the siteto the new unreferenced three-dimensional representation built based onthe plurality of new aerial images of the site without georeferenceinformation to create a new georeferenced three-dimensionalrepresentation of the site.
 12. The system of claim 11, furthercomprising instructions that, when executed by the at least oneprocessor, cause the system to generate a georeferenced orthomap fromthe new georeferenced three-dimensional representation, wherein thegeoreferenced orthomap comprises a geometrically corrected aerial imageof the site where tilt and relief have been corrected to portray auniform scale.
 13. The system of claim 11, wherein the georeferenceinformation for the site comprises known ground control points portrayedin the first plurality of aerial images.
 14. The system of claim 11,wherein the georeference information for the site comprises informationobtained from a global positioning system during a period of time when adigital camera captured the first plurality of aerial images.
 15. Thesystem of claim 14, wherein the plurality of new aerial images of thesite lack location information from the global positioning system.
 16. Anon-transitory computer readable medium storing instructions thereonthat, when executed by at least one processor, cause a computer systemto: build an initial three-dimensional representation of a site from afirst plurality of aerial images of the site, the first plurality ofaerial images comprising georeference information for the site; generatea transformation based on the initial three-dimensional representationof the site and the georeference information for the site; access aplurality of new aerial images of the site without georeferenceinformation; build a new unreferenced three-dimensional representationof the site based on the plurality of new aerial images of the sitewithout georeference information; and apply the transformation generatedbased on the initial three-dimensional representation of the site andthe georeference information for the site to the new unreferencedthree-dimensional representation built based on the plurality of newaerial images of the site without georeference information to create anew georeferenced three-dimensional representation of the site.
 17. Thenon-transitory computer readable medium of claim 16, further comprisinginstructions that, when executed by the at least one processor, causethe computer system to generate a georeferenced orthomap from the newgeoreferenced three-dimensional representation, wherein thegeoreferenced orthomap comprises a geometrically corrected aerial imageof the site where tilt and relief have been corrected to portray auniform scale.
 18. The non-transitory computer readable medium of claim16, wherein the georeference information for the site comprises knownground control points portrayed in the first plurality of aerial images.19. The non-transitory computer readable medium of claim 16, wherein thegeoreference information for the site comprises information obtainedfrom a global positioning system during a period of time when a digitalcamera captured the first plurality of aerial images.
 20. Thenon-transitory computer readable medium of claim 19, wherein theplurality of new aerial images of the site lack location informationfrom the global positioning system.